README.md

---
license: cc-by-nc-4.0
tags:
- mteb
model-index:
- name: text_sonar_basic_encoder_normalized
  results:
  - task:
      type: Clustering
    dataset:
      type: PL-MTEB/8tags-clustering
      name: MTEB 8TagsClustering
      config: default
      split: test
      revision: None
    metrics:
    - type: v_measure
      value: 18.787544117314575
  - task:
      type: STS
    dataset:
      type: C-MTEB/AFQMC
      name: MTEB AFQMC
      config: default
      split: validation
      revision: b44c3b011063adb25877c13823db83bb193913c4
    metrics:
    - type: cos_sim_pearson
      value: 17.97026675319667
    - type: cos_sim_spearman
      value: 17.63407829948615
    - type: euclidean_pearson
      value: 17.704571608660725
    - type: euclidean_spearman
      value: 17.634078298828143
    - type: manhattan_pearson
      value: 17.606959101509464
    - type: manhattan_spearman
      value: 17.549620164990085
  - task:
      type: STS
    dataset:
      type: C-MTEB/ATEC
      name: MTEB ATEC
      config: default
      split: test
      revision: 0f319b1142f28d00e055a6770f3f726ae9b7d865
    metrics:
    - type: cos_sim_pearson
      value: 27.670887504789675
    - type: cos_sim_spearman
      value: 26.176629407301782
    - type: euclidean_pearson
      value: 28.878485717935586
    - type: euclidean_spearman
      value: 26.176635036613355
    - type: manhattan_pearson
      value: 28.782373978690103
    - type: manhattan_spearman
      value: 26.055266444113794
  - task:
      type: Classification
    dataset:
      type: PL-MTEB/allegro-reviews
      name: MTEB AllegroReviews
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 29.62226640159046
    - type: f1
      value: 27.632722290701047
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_counterfactual
      name: MTEB AmazonCounterfactualClassification (en)
      config: en
      split: test
      revision: e8379541af4e31359cca9fbcf4b00f2671dba205
    metrics:
    - type: accuracy
      value: 81.49253731343285
    - type: ap
      value: 46.61440947240349
    - type: f1
      value: 75.68925212232107
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_counterfactual
      name: MTEB AmazonCounterfactualClassification (de)
      config: de
      split: test
      revision: e8379541af4e31359cca9fbcf4b00f2671dba205
    metrics:
    - type: accuracy
      value: 72.02355460385438
    - type: ap
      value: 83.13664983282676
    - type: f1
      value: 70.48997817871013
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_counterfactual
      name: MTEB AmazonCounterfactualClassification (en-ext)
      config: en-ext
      split: test
      revision: e8379541af4e31359cca9fbcf4b00f2671dba205
    metrics:
    - type: accuracy
      value: 82.09145427286357
    - type: ap
      value: 31.45181004731995
    - type: f1
      value: 69.41750580313406
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_counterfactual
      name: MTEB AmazonCounterfactualClassification (ja)
      config: ja
      split: test
      revision: e8379541af4e31359cca9fbcf4b00f2671dba205
    metrics:
    - type: accuracy
      value: 71.78800856531049
    - type: ap
      value: 19.65443896353892
    - type: f1
      value: 58.436688187826334
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_polarity
      name: MTEB AmazonPolarityClassification
      config: default
      split: test
      revision: e2d317d38cd51312af73b3d32a06d1a08b442046
    metrics:
    - type: accuracy
      value: 62.73074999999999
    - type: ap
      value: 58.2839375458089
    - type: f1
      value: 62.16204082406629
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_reviews_multi
      name: MTEB AmazonReviewsClassification (en)
      config: en
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 31.552000000000003
    - type: f1
      value: 31.125328770568277
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_reviews_multi
      name: MTEB AmazonReviewsClassification (de)
      config: de
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 34.611999999999995
    - type: f1
      value: 33.93738697105999
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_reviews_multi
      name: MTEB AmazonReviewsClassification (es)
      config: es
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 35.172
    - type: f1
      value: 34.14112656493798
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_reviews_multi
      name: MTEB AmazonReviewsClassification (fr)
      config: fr
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 34.910000000000004
    - type: f1
      value: 34.276631172288965
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_reviews_multi
      name: MTEB AmazonReviewsClassification (ja)
      config: ja
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 31.844
    - type: f1
      value: 31.478780923476368
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_reviews_multi
      name: MTEB AmazonReviewsClassification (zh)
      config: zh
      split: test
      revision: 1399c76144fd37290681b995c656ef9b2e06e26d
    metrics:
    - type: accuracy
      value: 31.912000000000003
    - type: f1
      value: 31.384992191831312
  - task:
      type: Classification
    dataset:
      type: DDSC/angry-tweets
      name: MTEB AngryTweetsClassification
      config: default
      split: test
      revision: 20b0e6081892e78179356fada741b7afa381443d
    metrics:
    - type: accuracy
      value: 49.61795606494747
    - type: f1
      value: 48.63625944670304
  - task:
      type: Retrieval
    dataset:
      type: arguana
      name: MTEB ArguAna
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 8.677
    - type: map_at_10
      value: 14.732000000000001
    - type: map_at_100
      value: 15.501999999999999
    - type: map_at_1000
      value: 15.583
    - type: map_at_3
      value: 12.553
    - type: map_at_5
      value: 13.822999999999999
    - type: mrr_at_1
      value: 8.819
    - type: mrr_at_10
      value: 14.787
    - type: mrr_at_100
      value: 15.557000000000002
    - type: mrr_at_1000
      value: 15.638
    - type: mrr_at_3
      value: 12.648000000000001
    - type: mrr_at_5
      value: 13.879
    - type: ndcg_at_1
      value: 8.677
    - type: ndcg_at_10
      value: 18.295
    - type: ndcg_at_100
      value: 22.353
    - type: ndcg_at_1000
      value: 24.948999999999998
    - type: ndcg_at_3
      value: 13.789000000000001
    - type: ndcg_at_5
      value: 16.075
    - type: precision_at_1
      value: 8.677
    - type: precision_at_10
      value: 2.98
    - type: precision_at_100
      value: 0.49500000000000005
    - type: precision_at_1000
      value: 0.07100000000000001
    - type: precision_at_3
      value: 5.785
    - type: precision_at_5
      value: 4.58
    - type: recall_at_1
      value: 8.677
    - type: recall_at_10
      value: 29.801
    - type: recall_at_100
      value: 49.502
    - type: recall_at_1000
      value: 70.91
    - type: recall_at_3
      value: 17.354
    - type: recall_at_5
      value: 22.902
  - task:
      type: Retrieval
    dataset:
      type: arguana-pl
      name: MTEB ArguAna-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 7.752000000000001
    - type: map_at_10
      value: 12.248000000000001
    - type: map_at_100
      value: 12.882
    - type: map_at_1000
      value: 12.963
    - type: map_at_3
      value: 10.574
    - type: map_at_5
      value: 11.566
    - type: mrr_at_1
      value: 7.824000000000001
    - type: mrr_at_10
      value: 12.293
    - type: mrr_at_100
      value: 12.928
    - type: mrr_at_1000
      value: 13.008000000000001
    - type: mrr_at_3
      value: 10.586
    - type: mrr_at_5
      value: 11.599
    - type: ndcg_at_1
      value: 7.752000000000001
    - type: ndcg_at_10
      value: 15.035000000000002
    - type: ndcg_at_100
      value: 18.497
    - type: ndcg_at_1000
      value: 20.896
    - type: ndcg_at_3
      value: 11.578
    - type: ndcg_at_5
      value: 13.38
    - type: precision_at_1
      value: 7.752000000000001
    - type: precision_at_10
      value: 2.404
    - type: precision_at_100
      value: 0.411
    - type: precision_at_1000
      value: 0.061
    - type: precision_at_3
      value: 4.836
    - type: precision_at_5
      value: 3.784
    - type: recall_at_1
      value: 7.752000000000001
    - type: recall_at_10
      value: 24.04
    - type: recall_at_100
      value: 41.11
    - type: recall_at_1000
      value: 60.597
    - type: recall_at_3
      value: 14.509
    - type: recall_at_5
      value: 18.919
  - task:
      type: Clustering
    dataset:
      type: mteb/arxiv-clustering-p2p
      name: MTEB ArxivClusteringP2P
      config: default
      split: test
      revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
    metrics:
    - type: v_measure
      value: 26.81177290816682
  - task:
      type: Clustering
    dataset:
      type: mteb/arxiv-clustering-s2s
      name: MTEB ArxivClusteringS2S
      config: default
      split: test
      revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
    metrics:
    - type: v_measure
      value: 24.346811178757022
  - task:
      type: Reranking
    dataset:
      type: mteb/askubuntudupquestions-reranking
      name: MTEB AskUbuntuDupQuestions
      config: default
      split: test
      revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
    metrics:
    - type: map
      value: 50.88606427049027
    - type: mrr
      value: 65.13004001231148
  - task:
      type: STS
    dataset:
      type: mteb/biosses-sts
      name: MTEB BIOSSES
      config: default
      split: test
      revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
    metrics:
    - type: cos_sim_pearson
      value: 77.15058512395619
    - type: cos_sim_spearman
      value: 79.10541692841936
    - type: euclidean_pearson
      value: 75.30525535929353
    - type: euclidean_spearman
      value: 79.10541692841936
    - type: manhattan_pearson
      value: 75.33508042552984
    - type: manhattan_spearman
      value: 78.84577245802708
  - task:
      type: STS
    dataset:
      type: C-MTEB/BQ
      name: MTEB BQ
      config: default
      split: test
      revision: e3dda5e115e487b39ec7e618c0c6a29137052a55
    metrics:
    - type: cos_sim_pearson
      value: 37.84739189558895
    - type: cos_sim_spearman
      value: 37.662710610486265
    - type: euclidean_pearson
      value: 37.5407537185213
    - type: euclidean_spearman
      value: 37.66272446700578
    - type: manhattan_pearson
      value: 37.863820146709706
    - type: manhattan_spearman
      value: 38.09120266204032
  - task:
      type: BitextMining
    dataset:
      type: mteb/bucc-bitext-mining
      name: MTEB BUCC (de-en)
      config: de-en
      split: test
      revision: d51519689f32196a32af33b075a01d0e7c51e252
    metrics:
    - type: accuracy
      value: 98.97703549060543
    - type: f1
      value: 98.82393876130828
    - type: precision
      value: 98.74913013221992
    - type: recall
      value: 98.97703549060543
  - task:
      type: BitextMining
    dataset:
      type: mteb/bucc-bitext-mining
      name: MTEB BUCC (fr-en)
      config: fr-en
      split: test
      revision: d51519689f32196a32af33b075a01d0e7c51e252
    metrics:
    - type: accuracy
      value: 98.34910851860005
    - type: f1
      value: 98.09487123046446
    - type: precision
      value: 97.97032063981217
    - type: recall
      value: 98.34910851860005
  - task:
      type: BitextMining
    dataset:
      type: mteb/bucc-bitext-mining
      name: MTEB BUCC (ru-en)
      config: ru-en
      split: test
      revision: d51519689f32196a32af33b075a01d0e7c51e252
    metrics:
    - type: accuracy
      value: 97.60304814686526
    - type: f1
      value: 97.36520032328832
    - type: precision
      value: 97.24743101258517
    - type: recall
      value: 97.60304814686526
  - task:
      type: BitextMining
    dataset:
      type: mteb/bucc-bitext-mining
      name: MTEB BUCC (zh-en)
      config: zh-en
      split: test
      revision: d51519689f32196a32af33b075a01d0e7c51e252
    metrics:
    - type: accuracy
      value: 98.78883622959452
    - type: f1
      value: 98.71862383710724
    - type: precision
      value: 98.68351764086361
    - type: recall
      value: 98.78883622959452
  - task:
      type: Classification
    dataset:
      type: mteb/banking77
      name: MTEB Banking77Classification
      config: default
      split: test
      revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
    metrics:
    - type: accuracy
      value: 73.49675324675324
    - type: f1
      value: 72.88538992490979
  - task:
      type: Clustering
    dataset:
      type: jinaai/big-patent-clustering
      name: MTEB BigPatentClustering
      config: default
      split: test
      revision: 62d5330920bca426ce9d3c76ea914f15fc83e891
    metrics:
    - type: v_measure
      value: 6.801245618724224
  - task:
      type: Clustering
    dataset:
      type: mteb/biorxiv-clustering-p2p
      name: MTEB BiorxivClusteringP2P
      config: default
      split: test
      revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
    metrics:
    - type: v_measure
      value: 20.6156033971932
  - task:
      type: Clustering
    dataset:
      type: mteb/biorxiv-clustering-s2s
      name: MTEB BiorxivClusteringS2S
      config: default
      split: test
      revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
    metrics:
    - type: v_measure
      value: 19.077587707743156
  - task:
      type: Clustering
    dataset:
      type: slvnwhrl/blurbs-clustering-p2p
      name: MTEB BlurbsClusteringP2P
      config: default
      split: test
      revision: a2dd5b02a77de3466a3eaa98ae586b5610314496
    metrics:
    - type: v_measure
      value: 27.00349462858046
  - task:
      type: Clustering
    dataset:
      type: slvnwhrl/blurbs-clustering-s2s
      name: MTEB BlurbsClusteringS2S
      config: default
      split: test
      revision: 9bfff9a7f8f6dc6ffc9da71c48dd48b68696471d
    metrics:
    - type: v_measure
      value: 14.845348131791589
  - task:
      type: BitextMining
    dataset:
      type: strombergnlp/bornholmsk_parallel
      name: MTEB BornholmBitextMining
      config: default
      split: test
      revision: 3bc5cfb4ec514264fe2db5615fac9016f7251552
    metrics:
    - type: accuracy
      value: 54.0
    - type: f1
      value: 47.37026862026861
    - type: precision
      value: 45.0734126984127
    - type: recall
      value: 54.0
  - task:
      type: Classification
    dataset:
      type: PL-MTEB/cbd
      name: MTEB CBD
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 63.83000000000001
    - type: ap
      value: 18.511972946438764
    - type: f1
      value: 53.16787370496645
  - task:
      type: PairClassification
    dataset:
      type: PL-MTEB/cdsce-pairclassification
      name: MTEB CDSC-E
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 84.39999999999999
    - type: cos_sim_ap
      value: 59.968589741258036
    - type: cos_sim_f1
      value: 54.90909090909091
    - type: cos_sim_precision
      value: 41.94444444444444
    - type: cos_sim_recall
      value: 79.47368421052632
    - type: dot_accuracy
      value: 84.39999999999999
    - type: dot_ap
      value: 59.968589741258036
    - type: dot_f1
      value: 54.90909090909091
    - type: dot_precision
      value: 41.94444444444444
    - type: dot_recall
      value: 79.47368421052632
    - type: euclidean_accuracy
      value: 84.39999999999999
    - type: euclidean_ap
      value: 59.968589741258036
    - type: euclidean_f1
      value: 54.90909090909091
    - type: euclidean_precision
      value: 41.94444444444444
    - type: euclidean_recall
      value: 79.47368421052632
    - type: manhattan_accuracy
      value: 84.39999999999999
    - type: manhattan_ap
      value: 60.094893481041154
    - type: manhattan_f1
      value: 55.452865064695004
    - type: manhattan_precision
      value: 42.73504273504273
    - type: manhattan_recall
      value: 78.94736842105263
    - type: max_accuracy
      value: 84.39999999999999
    - type: max_ap
      value: 60.094893481041154
    - type: max_f1
      value: 55.452865064695004
  - task:
      type: STS
    dataset:
      type: PL-MTEB/cdscr-sts
      name: MTEB CDSC-R
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 83.8427417206754
    - type: cos_sim_spearman
      value: 85.76946319798301
    - type: euclidean_pearson
      value: 79.43901249477852
    - type: euclidean_spearman
      value: 85.76946319798301
    - type: manhattan_pearson
      value: 79.81046681362531
    - type: manhattan_spearman
      value: 86.24115514951988
  - task:
      type: Clustering
    dataset:
      type: C-MTEB/CLSClusteringP2P
      name: MTEB CLSClusteringP2P
      config: default
      split: test
      revision: 4b6227591c6c1a73bc76b1055f3b7f3588e72476
    metrics:
    - type: v_measure
      value: 27.432031859995952
  - task:
      type: Clustering
    dataset:
      type: C-MTEB/CLSClusteringS2S
      name: MTEB CLSClusteringS2S
      config: default
      split: test
      revision: e458b3f5414b62b7f9f83499ac1f5497ae2e869f
    metrics:
    - type: v_measure
      value: 28.32367305628197
  - task:
      type: Reranking
    dataset:
      type: C-MTEB/CMedQAv1-reranking
      name: MTEB CMedQAv1
      config: default
      split: test
      revision: 8d7f1e942507dac42dc58017c1a001c3717da7df
    metrics:
    - type: map
      value: 34.30720667137015
    - type: mrr
      value: 40.24416666666666
  - task:
      type: Reranking
    dataset:
      type: C-MTEB/CMedQAv2-reranking
      name: MTEB CMedQAv2
      config: default
      split: test
      revision: 23d186750531a14a0357ca22cd92d712fd512ea0
    metrics:
    - type: map
      value: 35.87700379259406
    - type: mrr
      value: 40.80206349206349
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackAndroidRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 7.655000000000001
    - type: map_at_10
      value: 11.681999999999999
    - type: map_at_100
      value: 12.464
    - type: map_at_1000
      value: 12.603
    - type: map_at_3
      value: 10.514
    - type: map_at_5
      value: 11.083
    - type: mrr_at_1
      value: 10.157
    - type: mrr_at_10
      value: 14.773
    - type: mrr_at_100
      value: 15.581999999999999
    - type: mrr_at_1000
      value: 15.68
    - type: mrr_at_3
      value: 13.519
    - type: mrr_at_5
      value: 14.049
    - type: ndcg_at_1
      value: 10.157
    - type: ndcg_at_10
      value: 14.527999999999999
    - type: ndcg_at_100
      value: 18.695999999999998
    - type: ndcg_at_1000
      value: 22.709
    - type: ndcg_at_3
      value: 12.458
    - type: ndcg_at_5
      value: 13.152
    - type: precision_at_1
      value: 10.157
    - type: precision_at_10
      value: 2.976
    - type: precision_at_100
      value: 0.634
    - type: precision_at_1000
      value: 0.131
    - type: precision_at_3
      value: 6.152
    - type: precision_at_5
      value: 4.378
    - type: recall_at_1
      value: 7.655000000000001
    - type: recall_at_10
      value: 20.105
    - type: recall_at_100
      value: 39.181
    - type: recall_at_1000
      value: 68.06400000000001
    - type: recall_at_3
      value: 14.033000000000001
    - type: recall_at_5
      value: 16.209
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackEnglishRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 3.2329999999999997
    - type: map_at_10
      value: 5.378
    - type: map_at_100
      value: 5.774
    - type: map_at_1000
      value: 5.863
    - type: map_at_3
      value: 4.598
    - type: map_at_5
      value: 4.9750000000000005
    - type: mrr_at_1
      value: 4.076
    - type: mrr_at_10
      value: 6.679
    - type: mrr_at_100
      value: 7.151000000000001
    - type: mrr_at_1000
      value: 7.24
    - type: mrr_at_3
      value: 5.722
    - type: mrr_at_5
      value: 6.2059999999999995
    - type: ndcg_at_1
      value: 4.076
    - type: ndcg_at_10
      value: 6.994
    - type: ndcg_at_100
      value: 9.366
    - type: ndcg_at_1000
      value: 12.181000000000001
    - type: ndcg_at_3
      value: 5.356000000000001
    - type: ndcg_at_5
      value: 6.008
    - type: precision_at_1
      value: 4.076
    - type: precision_at_10
      value: 1.459
    - type: precision_at_100
      value: 0.334
    - type: precision_at_1000
      value: 0.075
    - type: precision_at_3
      value: 2.718
    - type: precision_at_5
      value: 2.089
    - type: recall_at_1
      value: 3.2329999999999997
    - type: recall_at_10
      value: 10.749
    - type: recall_at_100
      value: 21.776
    - type: recall_at_1000
      value: 42.278999999999996
    - type: recall_at_3
      value: 6.146999999999999
    - type: recall_at_5
      value: 7.779999999999999
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackGamingRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 8.036
    - type: map_at_10
      value: 12.727
    - type: map_at_100
      value: 13.532
    - type: map_at_1000
      value: 13.653
    - type: map_at_3
      value: 11.15
    - type: map_at_5
      value: 11.965
    - type: mrr_at_1
      value: 9.404
    - type: mrr_at_10
      value: 14.493
    - type: mrr_at_100
      value: 15.274
    - type: mrr_at_1000
      value: 15.370000000000001
    - type: mrr_at_3
      value: 12.853
    - type: mrr_at_5
      value: 13.696
    - type: ndcg_at_1
      value: 9.404
    - type: ndcg_at_10
      value: 15.784
    - type: ndcg_at_100
      value: 20.104
    - type: ndcg_at_1000
      value: 23.357
    - type: ndcg_at_3
      value: 12.61
    - type: ndcg_at_5
      value: 13.988
    - type: precision_at_1
      value: 9.404
    - type: precision_at_10
      value: 2.947
    - type: precision_at_100
      value: 0.562
    - type: precision_at_1000
      value: 0.093
    - type: precision_at_3
      value: 6.04
    - type: precision_at_5
      value: 4.4639999999999995
    - type: recall_at_1
      value: 8.036
    - type: recall_at_10
      value: 23.429
    - type: recall_at_100
      value: 43.728
    - type: recall_at_1000
      value: 68.10000000000001
    - type: recall_at_3
      value: 14.99
    - type: recall_at_5
      value: 18.274
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackGisRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 3.653
    - type: map_at_10
      value: 5.941
    - type: map_at_100
      value: 6.512
    - type: map_at_1000
      value: 6.6129999999999995
    - type: map_at_3
      value: 5.2540000000000004
    - type: map_at_5
      value: 5.645
    - type: mrr_at_1
      value: 3.955
    - type: mrr_at_10
      value: 6.4079999999999995
    - type: mrr_at_100
      value: 7.005999999999999
    - type: mrr_at_1000
      value: 7.105
    - type: mrr_at_3
      value: 5.593
    - type: mrr_at_5
      value: 6.051
    - type: ndcg_at_1
      value: 3.955
    - type: ndcg_at_10
      value: 7.342
    - type: ndcg_at_100
      value: 10.543
    - type: ndcg_at_1000
      value: 14.011000000000001
    - type: ndcg_at_3
      value: 5.853
    - type: ndcg_at_5
      value: 6.586
    - type: precision_at_1
      value: 3.955
    - type: precision_at_10
      value: 1.266
    - type: precision_at_100
      value: 0.315
    - type: precision_at_1000
      value: 0.066
    - type: precision_at_3
      value: 2.5989999999999998
    - type: precision_at_5
      value: 1.966
    - type: recall_at_1
      value: 3.653
    - type: recall_at_10
      value: 11.232000000000001
    - type: recall_at_100
      value: 26.625
    - type: recall_at_1000
      value: 54.476
    - type: recall_at_3
      value: 7.269
    - type: recall_at_5
      value: 8.982999999999999
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackMathematicaRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 2.257
    - type: map_at_10
      value: 3.881
    - type: map_at_100
      value: 4.279
    - type: map_at_1000
      value: 4.417
    - type: map_at_3
      value: 3.4070000000000005
    - type: map_at_5
      value: 3.744
    - type: mrr_at_1
      value: 2.9850000000000003
    - type: mrr_at_10
      value: 4.756
    - type: mrr_at_100
      value: 5.228
    - type: mrr_at_1000
      value: 5.354
    - type: mrr_at_3
      value: 4.125
    - type: mrr_at_5
      value: 4.567
    - type: ndcg_at_1
      value: 2.9850000000000003
    - type: ndcg_at_10
      value: 4.936999999999999
    - type: ndcg_at_100
      value: 7.664
    - type: ndcg_at_1000
      value: 12.045
    - type: ndcg_at_3
      value: 3.956
    - type: ndcg_at_5
      value: 4.584
    - type: precision_at_1
      value: 2.9850000000000003
    - type: precision_at_10
      value: 0.9329999999999999
    - type: precision_at_100
      value: 0.29
    - type: precision_at_1000
      value: 0.083
    - type: precision_at_3
      value: 1.949
    - type: precision_at_5
      value: 1.567
    - type: recall_at_1
      value: 2.257
    - type: recall_at_10
      value: 7.382
    - type: recall_at_100
      value: 20.689
    - type: recall_at_1000
      value: 53.586
    - type: recall_at_3
      value: 4.786
    - type: recall_at_5
      value: 6.2829999999999995
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackPhysicsRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 6.691
    - type: map_at_10
      value: 9.447
    - type: map_at_100
      value: 10.174
    - type: map_at_1000
      value: 10.308
    - type: map_at_3
      value: 8.187999999999999
    - type: map_at_5
      value: 8.852
    - type: mrr_at_1
      value: 8.566
    - type: mrr_at_10
      value: 12.036
    - type: mrr_at_100
      value: 12.817
    - type: mrr_at_1000
      value: 12.918
    - type: mrr_at_3
      value: 10.539
    - type: mrr_at_5
      value: 11.381
    - type: ndcg_at_1
      value: 8.566
    - type: ndcg_at_10
      value: 11.95
    - type: ndcg_at_100
      value: 15.831000000000001
    - type: ndcg_at_1000
      value: 19.561
    - type: ndcg_at_3
      value: 9.467
    - type: ndcg_at_5
      value: 10.544
    - type: precision_at_1
      value: 8.566
    - type: precision_at_10
      value: 2.387
    - type: precision_at_100
      value: 0.538
    - type: precision_at_1000
      value: 0.104
    - type: precision_at_3
      value: 4.556
    - type: precision_at_5
      value: 3.5029999999999997
    - type: recall_at_1
      value: 6.691
    - type: recall_at_10
      value: 17.375
    - type: recall_at_100
      value: 34.503
    - type: recall_at_1000
      value: 61.492000000000004
    - type: recall_at_3
      value: 10.134
    - type: recall_at_5
      value: 13.056999999999999
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackProgrammersRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 4.68
    - type: map_at_10
      value: 6.776999999999999
    - type: map_at_100
      value: 7.207
    - type: map_at_1000
      value: 7.321999999999999
    - type: map_at_3
      value: 6.007
    - type: map_at_5
      value: 6.356000000000001
    - type: mrr_at_1
      value: 5.479
    - type: mrr_at_10
      value: 8.094999999999999
    - type: mrr_at_100
      value: 8.622
    - type: mrr_at_1000
      value: 8.729000000000001
    - type: mrr_at_3
      value: 7.249
    - type: mrr_at_5
      value: 7.6770000000000005
    - type: ndcg_at_1
      value: 5.479
    - type: ndcg_at_10
      value: 8.474
    - type: ndcg_at_100
      value: 11.134
    - type: ndcg_at_1000
      value: 14.759
    - type: ndcg_at_3
      value: 6.888
    - type: ndcg_at_5
      value: 7.504
    - type: precision_at_1
      value: 5.479
    - type: precision_at_10
      value: 1.575
    - type: precision_at_100
      value: 0.35000000000000003
    - type: precision_at_1000
      value: 0.08099999999999999
    - type: precision_at_3
      value: 3.272
    - type: precision_at_5
      value: 2.374
    - type: recall_at_1
      value: 4.68
    - type: recall_at_10
      value: 12.552
    - type: recall_at_100
      value: 24.91
    - type: recall_at_1000
      value: 52.019999999999996
    - type: recall_at_3
      value: 8.057
    - type: recall_at_5
      value: 9.629999999999999
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 4.741750000000001
    - type: map_at_10
      value: 7.103916666666667
    - type: map_at_100
      value: 7.656499999999998
    - type: map_at_1000
      value: 7.767583333333332
    - type: map_at_3
      value: 6.262416666666668
    - type: map_at_5
      value: 6.693916666666667
    - type: mrr_at_1
      value: 5.780583333333332
    - type: mrr_at_10
      value: 8.576333333333332
    - type: mrr_at_100
      value: 9.17975
    - type: mrr_at_1000
      value: 9.279083333333334
    - type: mrr_at_3
      value: 7.608833333333333
    - type: mrr_at_5
      value: 8.111333333333333
    - type: ndcg_at_1
      value: 5.780583333333332
    - type: ndcg_at_10
      value: 8.866166666666668
    - type: ndcg_at_100
      value: 12.037083333333333
    - type: ndcg_at_1000
      value: 15.4555
    - type: ndcg_at_3
      value: 7.179083333333335
    - type: ndcg_at_5
      value: 7.897166666666666
    - type: precision_at_1
      value: 5.780583333333332
    - type: precision_at_10
      value: 1.6935833333333334
    - type: precision_at_100
      value: 0.3921666666666667
    - type: precision_at_1000
      value: 0.08391666666666667
    - type: precision_at_3
      value: 3.425416666666666
    - type: precision_at_5
      value: 2.5570833333333334
    - type: recall_at_1
      value: 4.741750000000001
    - type: recall_at_10
      value: 12.889083333333334
    - type: recall_at_100
      value: 27.81866666666667
    - type: recall_at_1000
      value: 53.52316666666667
    - type: recall_at_3
      value: 8.179333333333332
    - type: recall_at_5
      value: 10.004083333333334
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackStatsRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 3.7130000000000005
    - type: map_at_10
      value: 5.734
    - type: map_at_100
      value: 6.297999999999999
    - type: map_at_1000
      value: 6.388000000000001
    - type: map_at_3
      value: 5.119
    - type: map_at_5
      value: 5.432
    - type: mrr_at_1
      value: 4.9079999999999995
    - type: mrr_at_10
      value: 7.2940000000000005
    - type: mrr_at_100
      value: 7.8549999999999995
    - type: mrr_at_1000
      value: 7.95
    - type: mrr_at_3
      value: 6.621
    - type: mrr_at_5
      value: 6.950000000000001
    - type: ndcg_at_1
      value: 4.9079999999999995
    - type: ndcg_at_10
      value: 7.167999999999999
    - type: ndcg_at_100
      value: 10.436
    - type: ndcg_at_1000
      value: 13.370999999999999
    - type: ndcg_at_3
      value: 5.959
    - type: ndcg_at_5
      value: 6.481000000000001
    - type: precision_at_1
      value: 4.9079999999999995
    - type: precision_at_10
      value: 1.3339999999999999
    - type: precision_at_100
      value: 0.33899999999999997
    - type: precision_at_1000
      value: 0.065
    - type: precision_at_3
      value: 2.965
    - type: precision_at_5
      value: 2.117
    - type: recall_at_1
      value: 3.7130000000000005
    - type: recall_at_10
      value: 10.156
    - type: recall_at_100
      value: 25.955000000000002
    - type: recall_at_1000
      value: 48.891
    - type: recall_at_3
      value: 6.795
    - type: recall_at_5
      value: 8.187999999999999
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackTexRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 2.114
    - type: map_at_10
      value: 3.4290000000000003
    - type: map_at_100
      value: 3.789
    - type: map_at_1000
      value: 3.878
    - type: map_at_3
      value: 2.9139999999999997
    - type: map_at_5
      value: 3.148
    - type: mrr_at_1
      value: 2.65
    - type: mrr_at_10
      value: 4.252000000000001
    - type: mrr_at_100
      value: 4.689
    - type: mrr_at_1000
      value: 4.782
    - type: mrr_at_3
      value: 3.671
    - type: mrr_at_5
      value: 3.9370000000000003
    - type: ndcg_at_1
      value: 2.65
    - type: ndcg_at_10
      value: 4.47
    - type: ndcg_at_100
      value: 6.654
    - type: ndcg_at_1000
      value: 9.713
    - type: ndcg_at_3
      value: 3.424
    - type: ndcg_at_5
      value: 3.794
    - type: precision_at_1
      value: 2.65
    - type: precision_at_10
      value: 0.9119999999999999
    - type: precision_at_100
      value: 0.248
    - type: precision_at_1000
      value: 0.063
    - type: precision_at_3
      value: 1.7209999999999999
    - type: precision_at_5
      value: 1.287
    - type: recall_at_1
      value: 2.114
    - type: recall_at_10
      value: 6.927
    - type: recall_at_100
      value: 17.26
    - type: recall_at_1000
      value: 40.672999999999995
    - type: recall_at_3
      value: 3.8859999999999997
    - type: recall_at_5
      value: 4.861
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackUnixRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 6.055
    - type: map_at_10
      value: 7.704999999999999
    - type: map_at_100
      value: 8.169
    - type: map_at_1000
      value: 8.257
    - type: map_at_3
      value: 7.033
    - type: map_at_5
      value: 7.4079999999999995
    - type: mrr_at_1
      value: 6.81
    - type: mrr_at_10
      value: 8.955
    - type: mrr_at_100
      value: 9.497
    - type: mrr_at_1000
      value: 9.583
    - type: mrr_at_3
      value: 8.116
    - type: mrr_at_5
      value: 8.526
    - type: ndcg_at_1
      value: 6.81
    - type: ndcg_at_10
      value: 9.113
    - type: ndcg_at_100
      value: 11.884
    - type: ndcg_at_1000
      value: 14.762
    - type: ndcg_at_3
      value: 7.675999999999999
    - type: ndcg_at_5
      value: 8.325000000000001
    - type: precision_at_1
      value: 6.81
    - type: precision_at_10
      value: 1.558
    - type: precision_at_100
      value: 0.34299999999999997
    - type: precision_at_1000
      value: 0.068
    - type: precision_at_3
      value: 3.2960000000000003
    - type: precision_at_5
      value: 2.388
    - type: recall_at_1
      value: 6.055
    - type: recall_at_10
      value: 12.219
    - type: recall_at_100
      value: 25.304
    - type: recall_at_1000
      value: 47.204
    - type: recall_at_3
      value: 8.387
    - type: recall_at_5
      value: 9.991
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackWebmastersRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 5.043
    - type: map_at_10
      value: 7.394
    - type: map_at_100
      value: 8.096
    - type: map_at_1000
      value: 8.243
    - type: map_at_3
      value: 6.300999999999999
    - type: map_at_5
      value: 6.7780000000000005
    - type: mrr_at_1
      value: 6.126
    - type: mrr_at_10
      value: 9.308
    - type: mrr_at_100
      value: 10.091
    - type: mrr_at_1000
      value: 10.206
    - type: mrr_at_3
      value: 7.938000000000001
    - type: mrr_at_5
      value: 8.64
    - type: ndcg_at_1
      value: 6.126
    - type: ndcg_at_10
      value: 9.474
    - type: ndcg_at_100
      value: 13.238
    - type: ndcg_at_1000
      value: 17.366
    - type: ndcg_at_3
      value: 7.3260000000000005
    - type: ndcg_at_5
      value: 8.167
    - type: precision_at_1
      value: 6.126
    - type: precision_at_10
      value: 1.9959999999999998
    - type: precision_at_100
      value: 0.494
    - type: precision_at_1000
      value: 0.125
    - type: precision_at_3
      value: 3.557
    - type: precision_at_5
      value: 2.9250000000000003
    - type: recall_at_1
      value: 5.043
    - type: recall_at_10
      value: 13.812
    - type: recall_at_100
      value: 31.375999999999998
    - type: recall_at_1000
      value: 61.309999999999995
    - type: recall_at_3
      value: 7.8020000000000005
    - type: recall_at_5
      value: 9.725999999999999
  - task:
      type: Retrieval
    dataset:
      type: BeIR/cqadupstack
      name: MTEB CQADupstackWordpressRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 3.771
    - type: map_at_10
      value: 5.152
    - type: map_at_100
      value: 5.584
    - type: map_at_1000
      value: 5.666
    - type: map_at_3
      value: 4.664
    - type: map_at_5
      value: 4.941
    - type: mrr_at_1
      value: 4.251
    - type: mrr_at_10
      value: 5.867
    - type: mrr_at_100
      value: 6.345000000000001
    - type: mrr_at_1000
      value: 6.432
    - type: mrr_at_3
      value: 5.36
    - type: mrr_at_5
      value: 5.656
    - type: ndcg_at_1
      value: 4.251
    - type: ndcg_at_10
      value: 6.16
    - type: ndcg_at_100
      value: 8.895
    - type: ndcg_at_1000
      value: 11.631
    - type: ndcg_at_3
      value: 5.176
    - type: ndcg_at_5
      value: 5.633
    - type: precision_at_1
      value: 4.251
    - type: precision_at_10
      value: 0.98
    - type: precision_at_100
      value: 0.259
    - type: precision_at_1000
      value: 0.053
    - type: precision_at_3
      value: 2.2800000000000002
    - type: precision_at_5
      value: 1.627
    - type: recall_at_1
      value: 3.771
    - type: recall_at_10
      value: 8.731
    - type: recall_at_100
      value: 22.517
    - type: recall_at_1000
      value: 44.183
    - type: recall_at_3
      value: 5.866
    - type: recall_at_5
      value: 7.066999999999999
  - task:
      type: Retrieval
    dataset:
      type: climate-fever
      name: MTEB ClimateFEVER
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.543
    - type: map_at_10
      value: 1.027
    - type: map_at_100
      value: 1.228
    - type: map_at_1000
      value: 1.266
    - type: map_at_3
      value: 0.756
    - type: map_at_5
      value: 0.877
    - type: mrr_at_1
      value: 1.3679999999999999
    - type: mrr_at_10
      value: 2.474
    - type: mrr_at_100
      value: 2.8369999999999997
    - type: mrr_at_1000
      value: 2.894
    - type: mrr_at_3
      value: 1.8780000000000001
    - type: mrr_at_5
      value: 2.1319999999999997
    - type: ndcg_at_1
      value: 1.3679999999999999
    - type: ndcg_at_10
      value: 1.791
    - type: ndcg_at_100
      value: 3.06
    - type: ndcg_at_1000
      value: 4.501
    - type: ndcg_at_3
      value: 1.16
    - type: ndcg_at_5
      value: 1.3419999999999999
    - type: precision_at_1
      value: 1.3679999999999999
    - type: precision_at_10
      value: 0.697
    - type: precision_at_100
      value: 0.193
    - type: precision_at_1000
      value: 0.045
    - type: precision_at_3
      value: 0.9339999999999999
    - type: precision_at_5
      value: 0.808
    - type: recall_at_1
      value: 0.543
    - type: recall_at_10
      value: 2.5149999999999997
    - type: recall_at_100
      value: 7.356999999999999
    - type: recall_at_1000
      value: 16.233
    - type: recall_at_3
      value: 1.018
    - type: recall_at_5
      value: 1.5150000000000001
  - task:
      type: Retrieval
    dataset:
      type: C-MTEB/CmedqaRetrieval
      name: MTEB CmedqaRetrieval
      config: default
      split: dev
      revision: cd540c506dae1cf9e9a59c3e06f42030d54e7301
    metrics:
    - type: map_at_1
      value: 3.7289999999999996
    - type: map_at_10
      value: 5.524
    - type: map_at_100
      value: 5.984
    - type: map_at_1000
      value: 6.087
    - type: map_at_3
      value: 4.854
    - type: map_at_5
      value: 5.2299999999999995
    - type: mrr_at_1
      value: 6.177
    - type: mrr_at_10
      value: 8.541
    - type: mrr_at_100
      value: 9.073
    - type: mrr_at_1000
      value: 9.161
    - type: mrr_at_3
      value: 7.71
    - type: mrr_at_5
      value: 8.148
    - type: ndcg_at_1
      value: 6.177
    - type: ndcg_at_10
      value: 7.217999999999999
    - type: ndcg_at_100
      value: 9.927
    - type: ndcg_at_1000
      value: 13.062000000000001
    - type: ndcg_at_3
      value: 6.0569999999999995
    - type: ndcg_at_5
      value: 6.544999999999999
    - type: precision_at_1
      value: 6.177
    - type: precision_at_10
      value: 1.6729999999999998
    - type: precision_at_100
      value: 0.38999999999999996
    - type: precision_at_1000
      value: 0.082
    - type: precision_at_3
      value: 3.5090000000000003
    - type: precision_at_5
      value: 2.596
    - type: recall_at_1
      value: 3.7289999999999996
    - type: recall_at_10
      value: 9.501
    - type: recall_at_100
      value: 21.444
    - type: recall_at_1000
      value: 43.891999999999996
    - type: recall_at_3
      value: 6.053
    - type: recall_at_5
      value: 7.531000000000001
  - task:
      type: PairClassification
    dataset:
      type: C-MTEB/CMNLI
      name: MTEB Cmnli
      config: default
      split: validation
      revision: 41bc36f332156f7adc9e38f53777c959b2ae9766
    metrics:
    - type: cos_sim_accuracy
      value: 58.123872519543
    - type: cos_sim_ap
      value: 61.86046509726734
    - type: cos_sim_f1
      value: 68.18181818181817
    - type: cos_sim_precision
      value: 52.4198617221873
    - type: cos_sim_recall
      value: 97.49824643441664
    - type: dot_accuracy
      value: 58.123872519543
    - type: dot_ap
      value: 61.860555259802986
    - type: dot_f1
      value: 68.18181818181817
    - type: dot_precision
      value: 52.4198617221873
    - type: dot_recall
      value: 97.49824643441664
    - type: euclidean_accuracy
      value: 58.123872519543
    - type: euclidean_ap
      value: 61.87698627731538
    - type: euclidean_f1
      value: 68.18181818181817
    - type: euclidean_precision
      value: 52.4198617221873
    - type: euclidean_recall
      value: 97.49824643441664
    - type: manhattan_accuracy
      value: 58.123872519543
    - type: manhattan_ap
      value: 61.99468883207791
    - type: manhattan_f1
      value: 68.33675564681727
    - type: manhattan_precision
      value: 52.671562420866046
    - type: manhattan_recall
      value: 97.26443768996961
    - type: max_accuracy
      value: 58.123872519543
    - type: max_ap
      value: 61.99468883207791
    - type: max_f1
      value: 68.33675564681727
  - task:
      type: Retrieval
    dataset:
      type: C-MTEB/CovidRetrieval
      name: MTEB CovidRetrieval
      config: default
      split: dev
      revision: 1271c7809071a13532e05f25fb53511ffce77117
    metrics:
    - type: map_at_1
      value: 6.428000000000001
    - type: map_at_10
      value: 8.883000000000001
    - type: map_at_100
      value: 9.549000000000001
    - type: map_at_1000
      value: 9.665
    - type: map_at_3
      value: 8.061
    - type: map_at_5
      value: 8.475000000000001
    - type: mrr_at_1
      value: 6.428000000000001
    - type: mrr_at_10
      value: 8.896999999999998
    - type: mrr_at_100
      value: 9.557
    - type: mrr_at_1000
      value: 9.674000000000001
    - type: mrr_at_3
      value: 8.061
    - type: mrr_at_5
      value: 8.488
    - type: ndcg_at_1
      value: 6.428000000000001
    - type: ndcg_at_10
      value: 10.382
    - type: ndcg_at_100
      value: 14.235999999999999
    - type: ndcg_at_1000
      value: 18.04
    - type: ndcg_at_3
      value: 8.613999999999999
    - type: ndcg_at_5
      value: 9.372
    - type: precision_at_1
      value: 6.428000000000001
    - type: precision_at_10
      value: 1.528
    - type: precision_at_100
      value: 0.349
    - type: precision_at_1000
      value: 0.067
    - type: precision_at_3
      value: 3.4070000000000005
    - type: precision_at_5
      value: 2.424
    - type: recall_at_1
      value: 6.428000000000001
    - type: recall_at_10
      value: 15.226999999999999
    - type: recall_at_100
      value: 34.694
    - type: recall_at_1000
      value: 66.07
    - type: recall_at_3
      value: 10.221
    - type: recall_at_5
      value: 12.065
  - task:
      type: Retrieval
    dataset:
      type: dbpedia-entity
      name: MTEB DBPedia
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.541
    - type: map_at_10
      value: 1.1560000000000001
    - type: map_at_100
      value: 1.508
    - type: map_at_1000
      value: 1.598
    - type: map_at_3
      value: 0.918
    - type: map_at_5
      value: 0.992
    - type: mrr_at_1
      value: 9.5
    - type: mrr_at_10
      value: 13.446
    - type: mrr_at_100
      value: 13.935
    - type: mrr_at_1000
      value: 14.008999999999999
    - type: mrr_at_3
      value: 12.083
    - type: mrr_at_5
      value: 12.733
    - type: ndcg_at_1
      value: 5.75
    - type: ndcg_at_10
      value: 3.9210000000000003
    - type: ndcg_at_100
      value: 3.975
    - type: ndcg_at_1000
      value: 5.634
    - type: ndcg_at_3
      value: 4.87
    - type: ndcg_at_5
      value: 4.259
    - type: precision_at_1
      value: 9.5
    - type: precision_at_10
      value: 3.9
    - type: precision_at_100
      value: 1.015
    - type: precision_at_1000
      value: 0.297
    - type: precision_at_3
      value: 6.75
    - type: precision_at_5
      value: 5.25
    - type: recall_at_1
      value: 0.541
    - type: recall_at_10
      value: 2.228
    - type: recall_at_100
      value: 4.9430000000000005
    - type: recall_at_1000
      value: 11.661000000000001
    - type: recall_at_3
      value: 1.264
    - type: recall_at_5
      value: 1.4869999999999999
  - task:
      type: Classification
    dataset:
      type: DDSC/dkhate
      name: MTEB DKHateClassification
      config: default
      split: test
      revision: 59d12749a3c91a186063c7d729ec392fda94681c
    metrics:
    - type: accuracy
      value: 69.96960486322187
    - type: ap
      value: 91.23131906690253
    - type: f1
      value: 57.11872970138122
  - task:
      type: Classification
    dataset:
      type: AI-Sweden/SuperLim
      name: MTEB DalajClassification
      config: default
      split: test
      revision: 7ebf0b4caa7b2ae39698a889de782c09e6f5ee56
    metrics:
    - type: accuracy
      value: 49.75225225225225
    - type: ap
      value: 49.88223192425368
    - type: f1
      value: 49.55059044107012
  - task:
      type: Classification
    dataset:
      type: danish_political_comments
      name: MTEB DanishPoliticalCommentsClassification
      config: default
      split: train
      revision: edbb03726c04a0efab14fc8c3b8b79e4d420e5a1
    metrics:
    - type: accuracy
      value: 37.58534554537886
    - type: f1
      value: 33.99440115952713
  - task:
      type: Retrieval
    dataset:
      type: C-MTEB/DuRetrieval
      name: MTEB DuRetrieval
      config: default
      split: dev
      revision: a1a333e290fe30b10f3f56498e3a0d911a693ced
    metrics:
    - type: map_at_1
      value: 0.608
    - type: map_at_10
      value: 0.882
    - type: map_at_100
      value: 0.962
    - type: map_at_1000
      value: 1.028
    - type: map_at_3
      value: 0.749
    - type: map_at_5
      value: 0.8240000000000001
    - type: mrr_at_1
      value: 2.0500000000000003
    - type: mrr_at_10
      value: 2.796
    - type: mrr_at_100
      value: 2.983
    - type: mrr_at_1000
      value: 3.09
    - type: mrr_at_3
      value: 2.483
    - type: mrr_at_5
      value: 2.661
    - type: ndcg_at_1
      value: 2.0500000000000003
    - type: ndcg_at_10
      value: 1.435
    - type: ndcg_at_100
      value: 1.991
    - type: ndcg_at_1000
      value: 4.961
    - type: ndcg_at_3
      value: 1.428
    - type: ndcg_at_5
      value: 1.369
    - type: precision_at_1
      value: 2.0500000000000003
    - type: precision_at_10
      value: 0.5349999999999999
    - type: precision_at_100
      value: 0.127
    - type: precision_at_1000
      value: 0.086
    - type: precision_at_3
      value: 1.05
    - type: precision_at_5
      value: 0.84
    - type: recall_at_1
      value: 0.608
    - type: recall_at_10
      value: 1.54
    - type: recall_at_100
      value: 3.5069999999999997
    - type: recall_at_1000
      value: 20.531
    - type: recall_at_3
      value: 0.901
    - type: recall_at_5
      value: 1.168
  - task:
      type: Retrieval
    dataset:
      type: C-MTEB/EcomRetrieval
      name: MTEB EcomRetrieval
      config: default
      split: dev
      revision: 687de13dc7294d6fd9be10c6945f9e8fec8166b9
    metrics:
    - type: map_at_1
      value: 3.1
    - type: map_at_10
      value: 4.016
    - type: map_at_100
      value: 4.455
    - type: map_at_1000
      value: 4.579
    - type: map_at_3
      value: 3.567
    - type: map_at_5
      value: 3.8019999999999996
    - type: mrr_at_1
      value: 3.1
    - type: mrr_at_10
      value: 4.016
    - type: mrr_at_100
      value: 4.455
    - type: mrr_at_1000
      value: 4.579
    - type: mrr_at_3
      value: 3.567
    - type: mrr_at_5
      value: 3.8019999999999996
    - type: ndcg_at_1
      value: 3.1
    - type: ndcg_at_10
      value: 4.684
    - type: ndcg_at_100
      value: 7.284
    - type: ndcg_at_1000
      value: 11.689
    - type: ndcg_at_3
      value: 3.7289999999999996
    - type: ndcg_at_5
      value: 4.146
    - type: precision_at_1
      value: 3.1
    - type: precision_at_10
      value: 0.69
    - type: precision_at_100
      value: 0.202
    - type: precision_at_1000
      value: 0.056999999999999995
    - type: precision_at_3
      value: 1.4000000000000001
    - type: precision_at_5
      value: 1.04
    - type: recall_at_1
      value: 3.1
    - type: recall_at_10
      value: 6.9
    - type: recall_at_100
      value: 20.200000000000003
    - type: recall_at_1000
      value: 57.3
    - type: recall_at_3
      value: 4.2
    - type: recall_at_5
      value: 5.2
  - task:
      type: Classification
    dataset:
      type: mteb/emotion
      name: MTEB EmotionClassification
      config: default
      split: test
      revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
    metrics:
    - type: accuracy
      value: 38.285000000000004
    - type: f1
      value: 35.35979931355028
  - task:
      type: Retrieval
    dataset:
      type: fever
      name: MTEB FEVER
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.9249999999999999
    - type: map_at_10
      value: 1.311
    - type: map_at_100
      value: 1.363
    - type: map_at_1000
      value: 1.376
    - type: map_at_3
      value: 1.145
    - type: map_at_5
      value: 1.233
    - type: mrr_at_1
      value: 0.975
    - type: mrr_at_10
      value: 1.371
    - type: mrr_at_100
      value: 1.426
    - type: mrr_at_1000
      value: 1.439
    - type: mrr_at_3
      value: 1.195
    - type: mrr_at_5
      value: 1.286
    - type: ndcg_at_1
      value: 0.975
    - type: ndcg_at_10
      value: 1.5859999999999999
    - type: ndcg_at_100
      value: 1.8800000000000001
    - type: ndcg_at_1000
      value: 2.313
    - type: ndcg_at_3
      value: 1.229
    - type: ndcg_at_5
      value: 1.388
    - type: precision_at_1
      value: 0.975
    - type: precision_at_10
      value: 0.254
    - type: precision_at_100
      value: 0.041
    - type: precision_at_1000
      value: 0.008
    - type: precision_at_3
      value: 0.49
    - type: precision_at_5
      value: 0.375
    - type: recall_at_1
      value: 0.9249999999999999
    - type: recall_at_10
      value: 2.4250000000000003
    - type: recall_at_100
      value: 3.866
    - type: recall_at_1000
      value: 7.401000000000001
    - type: recall_at_3
      value: 1.4200000000000002
    - type: recall_at_5
      value: 1.81
  - task:
      type: Retrieval
    dataset:
      type: fiqa-pl
      name: MTEB FiQA-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.959
    - type: map_at_10
      value: 1.952
    - type: map_at_100
      value: 2.281
    - type: map_at_1000
      value: 2.393
    - type: map_at_3
      value: 1.703
    - type: map_at_5
      value: 1.8319999999999999
    - type: mrr_at_1
      value: 2.469
    - type: mrr_at_10
      value: 4.547
    - type: mrr_at_100
      value: 5.021
    - type: mrr_at_1000
      value: 5.1339999999999995
    - type: mrr_at_3
      value: 3.884
    - type: mrr_at_5
      value: 4.223
    - type: ndcg_at_1
      value: 2.469
    - type: ndcg_at_10
      value: 3.098
    - type: ndcg_at_100
      value: 5.177
    - type: ndcg_at_1000
      value: 8.889
    - type: ndcg_at_3
      value: 2.7119999999999997
    - type: ndcg_at_5
      value: 2.8000000000000003
    - type: precision_at_1
      value: 2.469
    - type: precision_at_10
      value: 1.065
    - type: precision_at_100
      value: 0.321
    - type: precision_at_1000
      value: 0.095
    - type: precision_at_3
      value: 2.109
    - type: precision_at_5
      value: 1.574
    - type: recall_at_1
      value: 0.959
    - type: recall_at_10
      value: 4.075
    - type: recall_at_100
      value: 12.487
    - type: recall_at_1000
      value: 36.854
    - type: recall_at_3
      value: 2.632
    - type: recall_at_5
      value: 3.231
  - task:
      type: Retrieval
    dataset:
      type: fiqa
      name: MTEB FiQA2018
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 1.032
    - type: map_at_10
      value: 1.8849999999999998
    - type: map_at_100
      value: 2.167
    - type: map_at_1000
      value: 2.266
    - type: map_at_3
      value: 1.609
    - type: map_at_5
      value: 1.7680000000000002
    - type: mrr_at_1
      value: 2.6229999999999998
    - type: mrr_at_10
      value: 4.479
    - type: mrr_at_100
      value: 4.92
    - type: mrr_at_1000
      value: 5.029999999999999
    - type: mrr_at_3
      value: 3.7289999999999996
    - type: mrr_at_5
      value: 4.138
    - type: ndcg_at_1
      value: 2.6229999999999998
    - type: ndcg_at_10
      value: 3.005
    - type: ndcg_at_100
      value: 5.01
    - type: ndcg_at_1000
      value: 8.312
    - type: ndcg_at_3
      value: 2.548
    - type: ndcg_at_5
      value: 2.735
    - type: precision_at_1
      value: 2.6229999999999998
    - type: precision_at_10
      value: 1.049
    - type: precision_at_100
      value: 0.31
    - type: precision_at_1000
      value: 0.089
    - type: precision_at_3
      value: 1.955
    - type: precision_at_5
      value: 1.574
    - type: recall_at_1
      value: 1.032
    - type: recall_at_10
      value: 3.888
    - type: recall_at_100
      value: 12.414
    - type: recall_at_1000
      value: 33.823
    - type: recall_at_3
      value: 2.37
    - type: recall_at_5
      value: 3.077
  - task:
      type: Retrieval
    dataset:
      type: jinaai/ger_da_lir
      name: MTEB GerDaLIR
      config: default
      split: test
      revision: 0bb47f1d73827e96964edb84dfe552f62f4fd5eb
    metrics:
    - type: map_at_1
      value: 0.542
    - type: map_at_10
      value: 0.8130000000000001
    - type: map_at_100
      value: 0.898
    - type: map_at_1000
      value: 0.9209999999999999
    - type: map_at_3
      value: 0.709
    - type: map_at_5
      value: 0.764
    - type: mrr_at_1
      value: 0.594
    - type: mrr_at_10
      value: 0.8880000000000001
    - type: mrr_at_100
      value: 0.9820000000000001
    - type: mrr_at_1000
      value: 1.008
    - type: mrr_at_3
      value: 0.774
    - type: mrr_at_5
      value: 0.832
    - type: ndcg_at_1
      value: 0.594
    - type: ndcg_at_10
      value: 1.0030000000000001
    - type: ndcg_at_100
      value: 1.537
    - type: ndcg_at_1000
      value: 2.4330000000000003
    - type: ndcg_at_3
      value: 0.782
    - type: ndcg_at_5
      value: 0.882
    - type: precision_at_1
      value: 0.594
    - type: precision_at_10
      value: 0.16999999999999998
    - type: precision_at_100
      value: 0.048
    - type: precision_at_1000
      value: 0.013
    - type: precision_at_3
      value: 0.33899999999999997
    - type: precision_at_5
      value: 0.255
    - type: recall_at_1
      value: 0.542
    - type: recall_at_10
      value: 1.533
    - type: recall_at_100
      value: 4.204
    - type: recall_at_1000
      value: 11.574
    - type: recall_at_3
      value: 0.932
    - type: recall_at_5
      value: 1.172
  - task:
      type: Retrieval
    dataset:
      type: deepset/germandpr
      name: MTEB GermanDPR
      config: default
      split: test
      revision: 5129d02422a66be600ac89cd3e8531b4f97d347d
    metrics:
    - type: map_at_1
      value: 25.561
    - type: map_at_10
      value: 38.873000000000005
    - type: map_at_100
      value: 40.004
    - type: map_at_1000
      value: 40.03
    - type: map_at_3
      value: 34.585
    - type: map_at_5
      value: 36.980000000000004
    - type: mrr_at_1
      value: 25.463
    - type: mrr_at_10
      value: 38.792
    - type: mrr_at_100
      value: 39.922000000000004
    - type: mrr_at_1000
      value: 39.949
    - type: mrr_at_3
      value: 34.504000000000005
    - type: mrr_at_5
      value: 36.899
    - type: ndcg_at_1
      value: 25.561
    - type: ndcg_at_10
      value: 46.477000000000004
    - type: ndcg_at_100
      value: 51.751999999999995
    - type: ndcg_at_1000
      value: 52.366
    - type: ndcg_at_3
      value: 37.645
    - type: ndcg_at_5
      value: 41.953
    - type: precision_at_1
      value: 25.561
    - type: precision_at_10
      value: 7.083
    - type: precision_at_100
      value: 0.9490000000000001
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_3
      value: 15.512
    - type: precision_at_5
      value: 11.395
    - type: recall_at_1
      value: 25.561
    - type: recall_at_10
      value: 70.829
    - type: recall_at_100
      value: 94.92699999999999
    - type: recall_at_1000
      value: 99.61
    - type: recall_at_3
      value: 46.537
    - type: recall_at_5
      value: 56.976000000000006
  - task:
      type: Retrieval
    dataset:
      type: mteb/germanquad-retrieval
      name: MTEB GermanQuAD-Retrieval
      config: default
      split: test
      revision: f5c87ae5a2e7a5106606314eef45255f03151bb3
    metrics:
    - type: map_at_1
      value: 53.539
    - type: map_at_10
      value: 65.144
    - type: map_at_100
      value: 65.627
    - type: map_at_1000
      value: 65.63900000000001
    - type: map_at_3
      value: 62.598
    - type: map_at_5
      value: 64.302
    - type: mrr_at_1
      value: 53.539
    - type: mrr_at_10
      value: 65.144
    - type: mrr_at_100
      value: 65.627
    - type: mrr_at_1000
      value: 65.63900000000001
    - type: mrr_at_3
      value: 62.598
    - type: mrr_at_5
      value: 64.302
    - type: ndcg_at_1
      value: 53.539
    - type: ndcg_at_10
      value: 70.602
    - type: ndcg_at_100
      value: 72.886
    - type: ndcg_at_1000
      value: 73.14500000000001
    - type: ndcg_at_3
      value: 65.52900000000001
    - type: ndcg_at_5
      value: 68.596
    - type: precision_at_1
      value: 53.539
    - type: precision_at_10
      value: 8.757
    - type: precision_at_100
      value: 0.9809999999999999
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_3
      value: 24.667
    - type: precision_at_5
      value: 16.289
    - type: recall_at_1
      value: 53.539
    - type: recall_at_10
      value: 87.568
    - type: recall_at_100
      value: 98.09400000000001
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_3
      value: 74.002
    - type: recall_at_5
      value: 81.443
  - task:
      type: STS
    dataset:
      type: jinaai/german-STSbenchmark
      name: MTEB GermanSTSBenchmark
      config: default
      split: test
      revision: e36907544d44c3a247898ed81540310442329e20
    metrics:
    - type: cos_sim_pearson
      value: 68.82052535790737
    - type: cos_sim_spearman
      value: 67.9356892072251
    - type: euclidean_pearson
      value: 67.2308663006278
    - type: euclidean_spearman
      value: 67.93572522920142
    - type: manhattan_pearson
      value: 67.23568952733595
    - type: manhattan_spearman
      value: 67.91660489262797
  - task:
      type: Retrieval
    dataset:
      type: hotpotqa
      name: MTEB HotpotQA
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 6.813
    - type: map_at_10
      value: 9.49
    - type: map_at_100
      value: 9.959
    - type: map_at_1000
      value: 10.024
    - type: map_at_3
      value: 8.618
    - type: map_at_5
      value: 9.084
    - type: mrr_at_1
      value: 13.626
    - type: mrr_at_10
      value: 17.818
    - type: mrr_at_100
      value: 18.412
    - type: mrr_at_1000
      value: 18.482000000000003
    - type: mrr_at_3
      value: 16.506999999999998
    - type: mrr_at_5
      value: 17.219
    - type: ndcg_at_1
      value: 13.626
    - type: ndcg_at_10
      value: 12.959999999999999
    - type: ndcg_at_100
      value: 15.562999999999999
    - type: ndcg_at_1000
      value: 17.571
    - type: ndcg_at_3
      value: 10.995000000000001
    - type: ndcg_at_5
      value: 11.908000000000001
    - type: precision_at_1
      value: 13.626
    - type: precision_at_10
      value: 2.995
    - type: precision_at_100
      value: 0.51
    - type: precision_at_1000
      value: 0.078
    - type: precision_at_3
      value: 7.000000000000001
    - type: precision_at_5
      value: 4.926
    - type: recall_at_1
      value: 6.813
    - type: recall_at_10
      value: 14.976
    - type: recall_at_100
      value: 25.517
    - type: recall_at_1000
      value: 39.095
    - type: recall_at_3
      value: 10.5
    - type: recall_at_5
      value: 12.316
  - task:
      type: Classification
    dataset:
      type: C-MTEB/IFlyTek-classification
      name: MTEB IFlyTek
      config: default
      split: validation
      revision: 421605374b29664c5fc098418fe20ada9bd55f8a
    metrics:
    - type: accuracy
      value: 38.01462100808003
    - type: f1
      value: 26.680357453754215
  - task:
      type: Classification
    dataset:
      type: mteb/imdb
      name: MTEB ImdbClassification
      config: default
      split: test
      revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
    metrics:
    - type: accuracy
      value: 55.7508
    - type: ap
      value: 53.28158993124153
    - type: f1
      value: 55.34571379744637
  - task:
      type: Classification
    dataset:
      type: C-MTEB/JDReview-classification
      name: MTEB JDReview
      config: default
      split: test
      revision: b7c64bd89eb87f8ded463478346f76731f07bf8b
    metrics:
    - type: accuracy
      value: 69.58724202626641
    - type: ap
      value: 30.04577466931377
    - type: f1
      value: 62.46921898313143
  - task:
      type: STS
    dataset:
      type: C-MTEB/LCQMC
      name: MTEB LCQMC
      config: default
      split: test
      revision: 17f9b096f80380fce5ed12a9be8be7784b337daf
    metrics:
    - type: cos_sim_pearson
      value: 48.80585861169271
    - type: cos_sim_spearman
      value: 50.11025991147549
    - type: euclidean_pearson
      value: 50.055425341198934
    - type: euclidean_spearman
      value: 50.11024862622995
    - type: manhattan_pearson
      value: 50.029980024931064
    - type: manhattan_spearman
      value: 50.074388245963384
  - task:
      type: Classification
    dataset:
      type: DDSC/lcc
      name: MTEB LccSentimentClassification
      config: default
      split: test
      revision: de7ba3406ee55ea2cc52a0a41408fa6aede6d3c6
    metrics:
    - type: accuracy
      value: 54.266666666666666
    - type: f1
      value: 52.181931818742875
  - task:
      type: Reranking
    dataset:
      type: jinaai/miracl
      name: MTEB MIRACL
      config: default
      split: test
      revision: d28a029f35c4ff7f616df47b0edf54e6882395e6
    metrics:
    - type: map
      value: 51.40745004398599
    - type: mrr
      value: 56.71940267335004
  - task:
      type: Reranking
    dataset:
      type: C-MTEB/Mmarco-reranking
      name: MTEB MMarcoReranking
      config: default
      split: dev
      revision: 8e0c766dbe9e16e1d221116a3f36795fbade07f6
    metrics:
    - type: map
      value: 5.831060174627054
    - type: mrr
      value: 4.019047619047618
  - task:
      type: Retrieval
    dataset:
      type: C-MTEB/MMarcoRetrieval
      name: MTEB MMarcoRetrieval
      config: default
      split: dev
      revision: 539bbde593d947e2a124ba72651aafc09eb33fc2
    metrics:
    - type: map_at_1
      value: 5.826
    - type: map_at_10
      value: 8.956999999999999
    - type: map_at_100
      value: 9.746
    - type: map_at_1000
      value: 9.873999999999999
    - type: map_at_3
      value: 7.757
    - type: map_at_5
      value: 8.373
    - type: mrr_at_1
      value: 6.046
    - type: mrr_at_10
      value: 9.251
    - type: mrr_at_100
      value: 10.044
    - type: mrr_at_1000
      value: 10.167
    - type: mrr_at_3
      value: 8.028
    - type: mrr_at_5
      value: 8.66
    - type: ndcg_at_1
      value: 6.046
    - type: ndcg_at_10
      value: 10.998
    - type: ndcg_at_100
      value: 15.568999999999999
    - type: ndcg_at_1000
      value: 19.453
    - type: ndcg_at_3
      value: 8.468
    - type: ndcg_at_5
      value: 9.582
    - type: precision_at_1
      value: 6.046
    - type: precision_at_10
      value: 1.807
    - type: precision_at_100
      value: 0.42500000000000004
    - type: precision_at_1000
      value: 0.076
    - type: precision_at_3
      value: 3.572
    - type: precision_at_5
      value: 2.702
    - type: recall_at_1
      value: 5.826
    - type: recall_at_10
      value: 17.291
    - type: recall_at_100
      value: 40.037
    - type: recall_at_1000
      value: 71.351
    - type: recall_at_3
      value: 10.269
    - type: recall_at_5
      value: 12.950000000000001
  - task:
      type: Retrieval
    dataset:
      type: msmarco
      name: MTEB MSMARCO
      config: default
      split: dev
      revision: None
    metrics:
    - type: map_at_1
      value: 1.203
    - type: map_at_10
      value: 2.27
    - type: map_at_100
      value: 2.5860000000000003
    - type: map_at_1000
      value: 2.661
    - type: map_at_3
      value: 1.8159999999999998
    - type: map_at_5
      value: 2.037
    - type: mrr_at_1
      value: 1.232
    - type: mrr_at_10
      value: 2.338
    - type: mrr_at_100
      value: 2.665
    - type: mrr_at_1000
      value: 2.7390000000000003
    - type: mrr_at_3
      value: 1.87
    - type: mrr_at_5
      value: 2.1
    - type: ndcg_at_1
      value: 1.232
    - type: ndcg_at_10
      value: 3.005
    - type: ndcg_at_100
      value: 4.936
    - type: ndcg_at_1000
      value: 7.441000000000001
    - type: ndcg_at_3
      value: 2.036
    - type: ndcg_at_5
      value: 2.435
    - type: precision_at_1
      value: 1.232
    - type: precision_at_10
      value: 0.549
    - type: precision_at_100
      value: 0.158
    - type: precision_at_1000
      value: 0.038
    - type: precision_at_3
      value: 0.903
    - type: precision_at_5
      value: 0.739
    - type: recall_at_1
      value: 1.203
    - type: recall_at_10
      value: 5.332
    - type: recall_at_100
      value: 15.164
    - type: recall_at_1000
      value: 35.831
    - type: recall_at_3
      value: 2.622
    - type: recall_at_5
      value: 3.572
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_domain
      name: MTEB MTOPDomainClassification (en)
      config: en
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 89.92476060191518
    - type: f1
      value: 89.30222882069823
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_domain
      name: MTEB MTOPDomainClassification (de)
      config: de
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 89.54353338968724
    - type: f1
      value: 88.23043644828002
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_domain
      name: MTEB MTOPDomainClassification (es)
      config: es
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 90.62374916611076
    - type: f1
      value: 89.68544977510335
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_domain
      name: MTEB MTOPDomainClassification (fr)
      config: fr
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 86.18540557469466
    - type: f1
      value: 85.7362674669331
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_domain
      name: MTEB MTOPDomainClassification (hi)
      config: hi
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 89.41556113302258
    - type: f1
      value: 89.04934651990581
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_domain
      name: MTEB MTOPDomainClassification (th)
      config: th
      split: test
      revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
    metrics:
    - type: accuracy
      value: 85.89511754068715
    - type: f1
      value: 85.57630467968119
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_intent
      name: MTEB MTOPIntentClassification (en)
      config: en
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 70.85043319653442
    - type: f1
      value: 46.0794069318026
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_intent
      name: MTEB MTOPIntentClassification (de)
      config: de
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 73.43195266272188
    - type: f1
      value: 48.08015719781981
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_intent
      name: MTEB MTOPIntentClassification (es)
      config: es
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 73.8425617078052
    - type: f1
      value: 49.37915156189611
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_intent
      name: MTEB MTOPIntentClassification (fr)
      config: fr
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 66.75227059191982
    - type: f1
      value: 43.4642946741452
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_intent
      name: MTEB MTOPIntentClassification (hi)
      config: hi
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 69.13589100035855
    - type: f1
      value: 46.25935961966482
  - task:
      type: Classification
    dataset:
      type: mteb/mtop_intent
      name: MTEB MTOPIntentClassification (th)
      config: th
      split: test
      revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
    metrics:
    - type: accuracy
      value: 68.47016274864377
    - type: f1
      value: 46.197113305277796
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (af)
      config: af
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 58.14727639542704
    - type: f1
      value: 55.58745169431752
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (am)
      config: am
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 57.91190316072628
    - type: f1
      value: 55.46589962622107
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ar)
      config: ar
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 57.22932078009414
    - type: f1
      value: 53.661218041561334
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (az)
      config: az
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 57.16543375924681
    - type: f1
      value: 55.16504653263189
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (bn)
      config: bn
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.239408204438476
    - type: f1
      value: 58.941991707183874
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (cy)
      config: cy
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 51.186953597848
    - type: f1
      value: 49.59432722397084
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (da)
      config: da
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.030934767989244
    - type: f1
      value: 58.836302050830966
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (de)
      config: de
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 61.314727639542696
    - type: f1
      value: 57.80700293522655
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (el)
      config: el
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 64.20645595158037
    - type: f1
      value: 61.36755812840151
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (en)
      config: en
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 64.36785474108943
    - type: f1
      value: 61.15645935863754
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (es)
      config: es
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 63.97108271687962
    - type: f1
      value: 62.07352472659557
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (fa)
      config: fa
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 66.67114996637525
    - type: f1
      value: 63.420170447126324
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (fi)
      config: fi
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.864828513786144
    - type: f1
      value: 59.655860488861926
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (fr)
      config: fr
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 58.55077336919974
    - type: f1
      value: 55.28215385204243
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (he)
      config: he
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 63.453261600538
    - type: f1
      value: 59.991998820039186
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (hi)
      config: hi
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 61.32145258910558
    - type: f1
      value: 58.9676667104426
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (hu)
      config: hu
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.905178211163424
    - type: f1
      value: 59.645126480791674
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (hy)
      config: hy
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 60.03026227303295
    - type: f1
      value: 56.68905593909442
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (id)
      config: id
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 63.28850033624749
    - type: f1
      value: 60.21862015326403
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (is)
      config: is
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 56.0221923335575
    - type: f1
      value: 53.388473451598315
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (it)
      config: it
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 64.44182918628111
    - type: f1
      value: 62.14806714489123
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ja)
      config: ja
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 63.69535978480162
    - type: f1
      value: 62.40231098840202
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (jv)
      config: jv
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 52.00067249495628
    - type: f1
      value: 48.871263427511984
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ka)
      config: ka
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 54.088769334229994
    - type: f1
      value: 52.68998451556
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (km)
      config: km
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 43.34229993275051
    - type: f1
      value: 40.578510490463024
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (kn)
      config: kn
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 57.87491593813046
    - type: f1
      value: 55.19579071673386
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ko)
      config: ko
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.69334229993275
    - type: f1
      value: 60.90210922623679
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (lv)
      config: lv
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 56.240753194351036
    - type: f1
      value: 54.137519761157485
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ml)
      config: ml
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.81439139206457
    - type: f1
      value: 60.46554841337619
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (mn)
      config: mn
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 58.49361129791527
    - type: f1
      value: 55.12919894175168
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ms)
      config: ms
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 61.55682582380633
    - type: f1
      value: 58.81763499302702
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (my)
      config: my
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 59.3981170141224
    - type: f1
      value: 56.31810441546048
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (nb)
      config: nb
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 59.89576328177538
    - type: f1
      value: 57.35130066022407
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (nl)
      config: nl
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 64.55951580363148
    - type: f1
      value: 61.50868742463585
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (pl)
      config: pl
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 65.86079354404842
    - type: f1
      value: 61.94702597578807
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (pt)
      config: pt
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 63.49024882313383
    - type: f1
      value: 60.796412851533454
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ro)
      config: ro
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.53194351042366
    - type: f1
      value: 59.9167382336848
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ru)
      config: ru
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.62945527908541
    - type: f1
      value: 59.195444230665096
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (sl)
      config: sl
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 63.43308675184935
    - type: f1
      value: 60.605749901316145
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (sq)
      config: sq
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 61.44586415601883
    - type: f1
      value: 58.635066561729396
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (sv)
      config: sv
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.86482851378615
    - type: f1
      value: 59.75440194153033
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (sw)
      config: sw
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 56.250840618695364
    - type: f1
      value: 54.84944007944625
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ta)
      config: ta
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 59.747814391392076
    - type: f1
      value: 56.83761137925043
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (te)
      config: te
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 59.60995292535306
    - type: f1
      value: 57.106776457430705
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (th)
      config: th
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 59.421654337592464
    - type: f1
      value: 57.81013790437749
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (tl)
      config: tl
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 58.120376597175515
    - type: f1
      value: 55.27690756097837
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (tr)
      config: tr
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 60.907868190988566
    - type: f1
      value: 57.43015543162361
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (ur)
      config: ur
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 59.492266308002705
    - type: f1
      value: 56.885590563156455
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (vi)
      config: vi
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 60.477471418964356
    - type: f1
      value: 57.87047944039945
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (zh-CN)
      config: zh-CN
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 62.07800941492938
    - type: f1
      value: 59.340232908410265
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_intent
      name: MTEB MassiveIntentClassification (zh-TW)
      config: zh-TW
      split: test
      revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
    metrics:
    - type: accuracy
      value: 56.73167451244117
    - type: f1
      value: 55.29236319279749
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (af)
      config: af
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 64.05850706119705
    - type: f1
      value: 62.20100273658395
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (am)
      config: am
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 63.24142568930733
    - type: f1
      value: 62.045023522098205
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ar)
      config: ar
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 63.685272360457304
    - type: f1
      value: 63.315744557403285
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (az)
      config: az
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 60.85743106926698
    - type: f1
      value: 59.106917986505636
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (bn)
      config: bn
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.1654337592468
    - type: f1
      value: 65.66986920813582
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (cy)
      config: cy
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 56.519838601210495
    - type: f1
      value: 54.73278620356587
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (da)
      config: da
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.76395427034298
    - type: f1
      value: 66.3447645997219
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (de)
      config: de
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.47814391392065
    - type: f1
      value: 66.32841368787447
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (el)
      config: el
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 70.22864828513787
    - type: f1
      value: 69.02774052818218
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (en)
      config: en
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 69.04841963685273
    - type: f1
      value: 67.70789401248665
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (es)
      config: es
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 69.08204438466711
    - type: f1
      value: 68.39277940460933
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (fa)
      config: fa
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 72.10154673839946
    - type: f1
      value: 70.7737194288215
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (fi)
      config: fi
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.16207128446537
    - type: f1
      value: 66.2311820377212
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (fr)
      config: fr
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 63.019502353732335
    - type: f1
      value: 62.105500895318656
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (he)
      config: he
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 68.82985877605918
    - type: f1
      value: 67.4894449433449
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (hi)
      config: hi
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 66.89643577673168
    - type: f1
      value: 65.45745898521055
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (hu)
      config: hu
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 69.32750504371216
    - type: f1
      value: 68.19665323990438
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (hy)
      config: hy
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 65.8238063214526
    - type: f1
      value: 64.60872984606974
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (id)
      config: id
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 68.98117014122394
    - type: f1
      value: 67.66697147027641
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (is)
      config: is
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 63.137188971082715
    - type: f1
      value: 61.58358081191463
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (it)
      config: it
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 70.0437121721587
    - type: f1
      value: 69.06747206775307
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ja)
      config: ja
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 70.67585743106926
    - type: f1
      value: 70.08618915891508
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (jv)
      config: jv
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 59.788164088769335
    - type: f1
      value: 57.91398932676417
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ka)
      config: ka
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 61.03227975790182
    - type: f1
      value: 60.044432258486715
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (km)
      config: km
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 49.051782111634154
    - type: f1
      value: 45.434581931581555
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (kn)
      config: kn
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 63.78278412911902
    - type: f1
      value: 62.106197625881535
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ko)
      config: ko
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 69.59986550100874
    - type: f1
      value: 68.94355682848476
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (lv)
      config: lv
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 59.97310020174847
    - type: f1
      value: 59.09912773329623
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ml)
      config: ml
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 69.20309347679893
    - type: f1
      value: 67.90665916607239
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (mn)
      config: mn
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 62.72024209818427
    - type: f1
      value: 60.77165334831407
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ms)
      config: ms
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.87155346334902
    - type: f1
      value: 65.7906032446679
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (my)
      config: my
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 64.97646267652992
    - type: f1
      value: 63.89390215791396
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (nb)
      config: nb
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 65.81371889710827
    - type: f1
      value: 64.39323436519936
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (nl)
      config: nl
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 69.79825151311366
    - type: f1
      value: 68.53789900442244
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (pl)
      config: pl
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 69.98991257565568
    - type: f1
      value: 68.93867074879778
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (pt)
      config: pt
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.50168123739071
    - type: f1
      value: 66.7457644903972
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ro)
      config: ro
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.52521856086078
    - type: f1
      value: 66.83370797374445
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ru)
      config: ru
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.96234028244787
    - type: f1
      value: 67.58983110064196
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (sl)
      config: sl
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 69.56624075319435
    - type: f1
      value: 68.35270162147211
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (sq)
      config: sq
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 68.48352387357095
    - type: f1
      value: 66.66973143886908
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (sv)
      config: sv
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.92535305985206
    - type: f1
      value: 66.52058462942483
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (sw)
      config: sw
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 63.184263618022875
    - type: f1
      value: 61.71153164960602
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ta)
      config: ta
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 64.8453261600538
    - type: f1
      value: 63.863209439112346
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (te)
      config: te
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 65.39340954942838
    - type: f1
      value: 63.85484524633183
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (th)
      config: th
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 67.9892400806994
    - type: f1
      value: 66.57022479007357
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (tl)
      config: tl
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 63.399462004034966
    - type: f1
      value: 61.62381473991175
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (tr)
      config: tr
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 65.773369199731
    - type: f1
      value: 65.58317907780943
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (ur)
      config: ur
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 65.8069939475454
    - type: f1
      value: 64.47027323557235
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (vi)
      config: vi
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 66.51647612642904
    - type: f1
      value: 65.66061210324213
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (zh-CN)
      config: zh-CN
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 68.88365837256221
    - type: f1
      value: 67.56956454874091
  - task:
      type: Classification
    dataset:
      type: mteb/amazon_massive_scenario
      name: MTEB MassiveScenarioClassification (zh-TW)
      config: zh-TW
      split: test
      revision: 7d571f92784cd94a019292a1f45445077d0ef634
    metrics:
    - type: accuracy
      value: 63.29858776059179
    - type: f1
      value: 62.76318771484755
  - task:
      type: Retrieval
    dataset:
      type: C-MTEB/MedicalRetrieval
      name: MTEB MedicalRetrieval
      config: default
      split: dev
      revision: 2039188fb5800a9803ba5048df7b76e6fb151fc6
    metrics:
    - type: map_at_1
      value: 2.9000000000000004
    - type: map_at_10
      value: 3.5360000000000005
    - type: map_at_100
      value: 3.703
    - type: map_at_1000
      value: 3.734
    - type: map_at_3
      value: 3.167
    - type: map_at_5
      value: 3.322
    - type: mrr_at_1
      value: 2.9000000000000004
    - type: mrr_at_10
      value: 3.5360000000000005
    - type: mrr_at_100
      value: 3.703
    - type: mrr_at_1000
      value: 3.734
    - type: mrr_at_3
      value: 3.167
    - type: mrr_at_5
      value: 3.322
    - type: ndcg_at_1
      value: 2.9000000000000004
    - type: ndcg_at_10
      value: 4.079
    - type: ndcg_at_100
      value: 5.101
    - type: ndcg_at_1000
      value: 6.295000000000001
    - type: ndcg_at_3
      value: 3.276
    - type: ndcg_at_5
      value: 3.56
    - type: precision_at_1
      value: 2.9000000000000004
    - type: precision_at_10
      value: 0.59
    - type: precision_at_100
      value: 0.11199999999999999
    - type: precision_at_1000
      value: 0.022000000000000002
    - type: precision_at_3
      value: 1.2
    - type: precision_at_5
      value: 0.86
    - type: recall_at_1
      value: 2.9000000000000004
    - type: recall_at_10
      value: 5.8999999999999995
    - type: recall_at_100
      value: 11.200000000000001
    - type: recall_at_1000
      value: 21.5
    - type: recall_at_3
      value: 3.5999999999999996
    - type: recall_at_5
      value: 4.3
  - task:
      type: Clustering
    dataset:
      type: mteb/medrxiv-clustering-p2p
      name: MTEB MedrxivClusteringP2P
      config: default
      split: test
      revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
    metrics:
    - type: v_measure
      value: 19.061819627060558
  - task:
      type: Clustering
    dataset:
      type: mteb/medrxiv-clustering-s2s
      name: MTEB MedrxivClusteringS2S
      config: default
      split: test
      revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
    metrics:
    - type: v_measure
      value: 19.79520446745267
  - task:
      type: Reranking
    dataset:
      type: mteb/mind_small
      name: MTEB MindSmallReranking
      config: default
      split: test
      revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
    metrics:
    - type: map
      value: 26.881162218991285
    - type: mrr
      value: 27.23201335662217
  - task:
      type: Classification
    dataset:
      type: C-MTEB/MultilingualSentiment-classification
      name: MTEB MultilingualSentiment
      config: default
      split: validation
      revision: 46958b007a63fdbf239b7672c25d0bea67b5ea1a
    metrics:
    - type: accuracy
      value: 57.69
    - type: f1
      value: 57.370451927892695
  - task:
      type: Retrieval
    dataset:
      type: nfcorpus
      name: MTEB NFCorpus
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.443
    - type: map_at_10
      value: 1.189
    - type: map_at_100
      value: 2.221
    - type: map_at_1000
      value: 3.034
    - type: map_at_3
      value: 0.683
    - type: map_at_5
      value: 0.882
    - type: mrr_at_1
      value: 4.334
    - type: mrr_at_10
      value: 10.908
    - type: mrr_at_100
      value: 12.536
    - type: mrr_at_1000
      value: 12.642000000000001
    - type: mrr_at_3
      value: 7.481999999999999
    - type: mrr_at_5
      value: 9.324
    - type: ndcg_at_1
      value: 3.7150000000000003
    - type: ndcg_at_10
      value: 5.591
    - type: ndcg_at_100
      value: 9.522
    - type: ndcg_at_1000
      value: 19.705000000000002
    - type: ndcg_at_3
      value: 4.292
    - type: ndcg_at_5
      value: 5.038
    - type: precision_at_1
      value: 4.334
    - type: precision_at_10
      value: 5.077
    - type: precision_at_100
      value: 3.2910000000000004
    - type: precision_at_1000
      value: 1.568
    - type: precision_at_3
      value: 4.644
    - type: precision_at_5
      value: 5.139
    - type: recall_at_1
      value: 0.443
    - type: recall_at_10
      value: 3.3520000000000003
    - type: recall_at_100
      value: 15.515
    - type: recall_at_1000
      value: 50.505
    - type: recall_at_3
      value: 0.931
    - type: recall_at_5
      value: 1.698
  - task:
      type: Retrieval
    dataset:
      type: nfcorpus-pl
      name: MTEB NFCorpus-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.307
    - type: map_at_10
      value: 0.835
    - type: map_at_100
      value: 1.503
    - type: map_at_1000
      value: 2.263
    - type: map_at_3
      value: 0.503
    - type: map_at_5
      value: 0.567
    - type: mrr_at_1
      value: 4.025
    - type: mrr_at_10
      value: 9.731
    - type: mrr_at_100
      value: 11.229
    - type: mrr_at_1000
      value: 11.34
    - type: mrr_at_3
      value: 6.811
    - type: mrr_at_5
      value: 8.126999999999999
    - type: ndcg_at_1
      value: 3.56
    - type: ndcg_at_10
      value: 4.596
    - type: ndcg_at_100
      value: 7.567
    - type: ndcg_at_1000
      value: 17.76
    - type: ndcg_at_3
      value: 3.52
    - type: ndcg_at_5
      value: 3.823
    - type: precision_at_1
      value: 4.025
    - type: precision_at_10
      value: 4.334
    - type: precision_at_100
      value: 2.842
    - type: precision_at_1000
      value: 1.506
    - type: precision_at_3
      value: 3.818
    - type: precision_at_5
      value: 4.149
    - type: recall_at_1
      value: 0.307
    - type: recall_at_10
      value: 2.543
    - type: recall_at_100
      value: 12.152000000000001
    - type: recall_at_1000
      value: 46.878
    - type: recall_at_3
      value: 0.755
    - type: recall_at_5
      value: 0.975
  - task:
      type: Retrieval
    dataset:
      type: nq
      name: MTEB NQ
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.439
    - type: map_at_10
      value: 0.6839999999999999
    - type: map_at_100
      value: 0.769
    - type: map_at_1000
      value: 0.79
    - type: map_at_3
      value: 0.584
    - type: map_at_5
      value: 0.621
    - type: mrr_at_1
      value: 0.5499999999999999
    - type: mrr_at_10
      value: 0.819
    - type: mrr_at_100
      value: 0.9169999999999999
    - type: mrr_at_1000
      value: 0.9400000000000001
    - type: mrr_at_3
      value: 0.705
    - type: mrr_at_5
      value: 0.75
    - type: ndcg_at_1
      value: 0.5499999999999999
    - type: ndcg_at_10
      value: 0.886
    - type: ndcg_at_100
      value: 1.422
    - type: ndcg_at_1000
      value: 2.2079999999999997
    - type: ndcg_at_3
      value: 0.6629999999999999
    - type: ndcg_at_5
      value: 0.735
    - type: precision_at_1
      value: 0.5499999999999999
    - type: precision_at_10
      value: 0.16199999999999998
    - type: precision_at_100
      value: 0.048
    - type: precision_at_1000
      value: 0.012
    - type: precision_at_3
      value: 0.309
    - type: precision_at_5
      value: 0.22599999999999998
    - type: recall_at_1
      value: 0.439
    - type: recall_at_10
      value: 1.405
    - type: recall_at_100
      value: 4.051
    - type: recall_at_1000
      value: 10.487
    - type: recall_at_3
      value: 0.787
    - type: recall_at_5
      value: 0.9560000000000001
  - task:
      type: Retrieval
    dataset:
      type: narrativeqa
      name: MTEB NarrativeQARetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 5.93
    - type: map_at_10
      value: 7.349
    - type: map_at_100
      value: 8.011
    - type: map_at_1000
      value: 8.351
    - type: map_at_3
      value: 6.787
    - type: map_at_5
      value: 7.02
    - type: mrr_at_1
      value: 5.93
    - type: mrr_at_10
      value: 7.349
    - type: mrr_at_100
      value: 8.011
    - type: mrr_at_1000
      value: 8.351
    - type: mrr_at_3
      value: 6.787
    - type: mrr_at_5
      value: 7.02
    - type: ndcg_at_1
      value: 5.93
    - type: ndcg_at_10
      value: 8.291
    - type: ndcg_at_100
      value: 12.833
    - type: ndcg_at_1000
      value: 21.253
    - type: ndcg_at_3
      value: 7.072000000000001
    - type: ndcg_at_5
      value: 7.495
    - type: precision_at_1
      value: 5.93
    - type: precision_at_10
      value: 1.1400000000000001
    - type: precision_at_100
      value: 0.359
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_3
      value: 2.633
    - type: precision_at_5
      value: 1.786
    - type: recall_at_1
      value: 5.93
    - type: recall_at_10
      value: 11.395
    - type: recall_at_100
      value: 35.929
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_3
      value: 7.9
    - type: recall_at_5
      value: 8.932
  - task:
      type: Classification
    dataset:
      type: ScandEval/norec-mini
      name: MTEB NoRecClassification
      config: default
      split: test
      revision: 07b99ab3363c2e7f8f87015b01c21f4d9b917ce3
    metrics:
    - type: accuracy
      value: 48.251953125
    - type: f1
      value: 45.42526611578402
  - task:
      type: Classification
    dataset:
      type: strombergnlp/nordic_langid
      name: MTEB NordicLangClassification
      config: default
      split: test
      revision: e254179d18ab0165fdb6dbef91178266222bee2a
    metrics:
    - type: accuracy
      value: 48.403333333333336
    - type: f1
      value: 47.9287124185198
  - task:
      type: BitextMining
    dataset:
      type: kardosdrur/norwegian-courts
      name: MTEB NorwegianCourtsBitextMining
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 93.85964912280701
    - type: f1
      value: 92.98245614035088
    - type: precision
      value: 92.54385964912281
    - type: recall
      value: 93.85964912280701
  - task:
      type: Classification
    dataset:
      type: NbAiLab/norwegian_parliament
      name: MTEB NorwegianParliament
      config: default
      split: test
      revision: f7393532774c66312378d30b197610b43d751972
    metrics:
    - type: accuracy
      value: 55.991666666666674
    - type: ap
      value: 53.417849849746226
    - type: f1
      value: 55.757916182475384
  - task:
      type: PairClassification
    dataset:
      type: C-MTEB/OCNLI
      name: MTEB Ocnli
      config: default
      split: validation
      revision: 66e76a618a34d6d565d5538088562851e6daa7ec
    metrics:
    - type: cos_sim_accuracy
      value: 54.68327016783974
    - type: cos_sim_ap
      value: 55.175059616546406
    - type: cos_sim_f1
      value: 67.81733189500179
    - type: cos_sim_precision
      value: 51.41766630316249
    - type: cos_sim_recall
      value: 99.57761351636748
    - type: dot_accuracy
      value: 54.68327016783974
    - type: dot_ap
      value: 55.175059616546406
    - type: dot_f1
      value: 67.81733189500179
    - type: dot_precision
      value: 51.41766630316249
    - type: dot_recall
      value: 99.57761351636748
    - type: euclidean_accuracy
      value: 54.68327016783974
    - type: euclidean_ap
      value: 55.17510180566365
    - type: euclidean_f1
      value: 67.81733189500179
    - type: euclidean_precision
      value: 51.41766630316249
    - type: euclidean_recall
      value: 99.57761351636748
    - type: manhattan_accuracy
      value: 55.44125609095831
    - type: manhattan_ap
      value: 55.76283671826867
    - type: manhattan_f1
      value: 68.05905653583004
    - type: manhattan_precision
      value: 51.63934426229508
    - type: manhattan_recall
      value: 99.78880675818374
    - type: max_accuracy
      value: 55.44125609095831
    - type: max_ap
      value: 55.76283671826867
    - type: max_f1
      value: 68.05905653583004
  - task:
      type: Classification
    dataset:
      type: C-MTEB/OnlineShopping-classification
      name: MTEB OnlineShopping
      config: default
      split: test
      revision: e610f2ebd179a8fda30ae534c3878750a96db120
    metrics:
    - type: accuracy
      value: 75.64
    - type: ap
      value: 71.45085103287833
    - type: f1
      value: 75.52254495697326
  - task:
      type: Classification
    dataset:
      type: laugustyniak/abusive-clauses-pl
      name: MTEB PAC
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 73.86620330147699
    - type: ap
      value: 80.58015815306322
    - type: f1
      value: 71.49082510883872
  - task:
      type: STS
    dataset:
      type: C-MTEB/PAWSX
      name: MTEB PAWSX
      config: default
      split: test
      revision: 9c6a90e430ac22b5779fb019a23e820b11a8b5e1
    metrics:
    - type: cos_sim_pearson
      value: 29.52361689421863
    - type: cos_sim_spearman
      value: 32.750058577257875
    - type: euclidean_pearson
      value: 34.583472972871796
    - type: euclidean_spearman
      value: 32.75328764421994
    - type: manhattan_pearson
      value: 34.727366510326995
    - type: manhattan_spearman
      value: 32.787167142114214
  - task:
      type: PairClassification
    dataset:
      type: PL-MTEB/ppc-pairclassification
      name: MTEB PPC
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 71.1
    - type: cos_sim_ap
      value: 85.36544548691205
    - type: cos_sim_f1
      value: 75.23393636930756
    - type: cos_sim_precision
      value: 60.36036036036037
    - type: cos_sim_recall
      value: 99.83443708609272
    - type: dot_accuracy
      value: 71.1
    - type: dot_ap
      value: 85.36544548691204
    - type: dot_f1
      value: 75.23393636930756
    - type: dot_precision
      value: 60.36036036036037
    - type: dot_recall
      value: 99.83443708609272
    - type: euclidean_accuracy
      value: 71.1
    - type: euclidean_ap
      value: 85.36544548691205
    - type: euclidean_f1
      value: 75.23393636930756
    - type: euclidean_precision
      value: 60.36036036036037
    - type: euclidean_recall
      value: 99.83443708609272
    - type: manhattan_accuracy
      value: 71.1
    - type: manhattan_ap
      value: 85.33853868545614
    - type: manhattan_f1
      value: 75.23393636930756
    - type: manhattan_precision
      value: 60.36036036036037
    - type: manhattan_recall
      value: 99.83443708609272
    - type: max_accuracy
      value: 71.1
    - type: max_ap
      value: 85.36544548691205
    - type: max_f1
      value: 75.23393636930756
  - task:
      type: PairClassification
    dataset:
      type: PL-MTEB/psc-pairclassification
      name: MTEB PSC
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 90.81632653061224
    - type: cos_sim_ap
      value: 91.97693749083473
    - type: cos_sim_f1
      value: 85.55078683834049
    - type: cos_sim_precision
      value: 80.59299191374663
    - type: cos_sim_recall
      value: 91.15853658536585
    - type: dot_accuracy
      value: 90.81632653061224
    - type: dot_ap
      value: 91.97693749083473
    - type: dot_f1
      value: 85.55078683834049
    - type: dot_precision
      value: 80.59299191374663
    - type: dot_recall
      value: 91.15853658536585
    - type: euclidean_accuracy
      value: 90.81632653061224
    - type: euclidean_ap
      value: 91.97693749083473
    - type: euclidean_f1
      value: 85.55078683834049
    - type: euclidean_precision
      value: 80.59299191374663
    - type: euclidean_recall
      value: 91.15853658536585
    - type: manhattan_accuracy
      value: 90.9090909090909
    - type: manhattan_ap
      value: 92.043441286281
    - type: manhattan_f1
      value: 85.34482758620689
    - type: manhattan_precision
      value: 80.70652173913044
    - type: manhattan_recall
      value: 90.54878048780488
    - type: max_accuracy
      value: 90.9090909090909
    - type: max_ap
      value: 92.043441286281
    - type: max_f1
      value: 85.55078683834049
  - task:
      type: PairClassification
    dataset:
      type: paws-x
      name: MTEB PawsX (de)
      config: de
      split: test
      revision: 8a04d940a42cd40658986fdd8e3da561533a3646
    metrics:
    - type: cos_sim_accuracy
      value: 70.35
    - type: cos_sim_ap
      value: 72.01641717127626
    - type: cos_sim_f1
      value: 64.49511400651467
    - type: cos_sim_precision
      value: 55.26315789473685
    - type: cos_sim_recall
      value: 77.43016759776536
    - type: dot_accuracy
      value: 70.35
    - type: dot_ap
      value: 72.06599137974572
    - type: dot_f1
      value: 64.49511400651467
    - type: dot_precision
      value: 55.26315789473685
    - type: dot_recall
      value: 77.43016759776536
    - type: euclidean_accuracy
      value: 70.35
    - type: euclidean_ap
      value: 71.92019289154159
    - type: euclidean_f1
      value: 64.49511400651467
    - type: euclidean_precision
      value: 55.26315789473685
    - type: euclidean_recall
      value: 77.43016759776536
    - type: manhattan_accuracy
      value: 70.35
    - type: manhattan_ap
      value: 71.92979188519502
    - type: manhattan_f1
      value: 64.60409019402202
    - type: manhattan_precision
      value: 60.86956521739131
    - type: manhattan_recall
      value: 68.8268156424581
    - type: max_accuracy
      value: 70.35
    - type: max_ap
      value: 72.06599137974572
    - type: max_f1
      value: 64.60409019402202
  - task:
      type: PairClassification
    dataset:
      type: paws-x
      name: MTEB PawsX (en)
      config: en
      split: test
      revision: 8a04d940a42cd40658986fdd8e3da561533a3646
    metrics:
    - type: cos_sim_accuracy
      value: 71.0
    - type: cos_sim_ap
      value: 74.73017292645147
    - type: cos_sim_f1
      value: 66.73427991886409
    - type: cos_sim_precision
      value: 61.78403755868545
    - type: cos_sim_recall
      value: 72.54685777287762
    - type: dot_accuracy
      value: 71.0
    - type: dot_ap
      value: 74.73017292645147
    - type: dot_f1
      value: 66.73427991886409
    - type: dot_precision
      value: 61.78403755868545
    - type: dot_recall
      value: 72.54685777287762
    - type: euclidean_accuracy
      value: 71.0
    - type: euclidean_ap
      value: 74.73013082197343
    - type: euclidean_f1
      value: 66.73427991886409
    - type: euclidean_precision
      value: 61.78403755868545
    - type: euclidean_recall
      value: 72.54685777287762
    - type: manhattan_accuracy
      value: 70.95
    - type: manhattan_ap
      value: 74.71203917486744
    - type: manhattan_f1
      value: 66.86868686868686
    - type: manhattan_precision
      value: 61.696178937558244
    - type: manhattan_recall
      value: 72.98787210584344
    - type: max_accuracy
      value: 71.0
    - type: max_ap
      value: 74.73017292645147
    - type: max_f1
      value: 66.86868686868686
  - task:
      type: PairClassification
    dataset:
      type: paws-x
      name: MTEB PawsX (es)
      config: es
      split: test
      revision: 8a04d940a42cd40658986fdd8e3da561533a3646
    metrics:
    - type: cos_sim_accuracy
      value: 67.7
    - type: cos_sim_ap
      value: 69.70320170421651
    - type: cos_sim_f1
      value: 62.55625562556255
    - type: cos_sim_precision
      value: 52.851711026615966
    - type: cos_sim_recall
      value: 76.62624035281146
    - type: dot_accuracy
      value: 67.7
    - type: dot_ap
      value: 69.70320170421651
    - type: dot_f1
      value: 62.55625562556255
    - type: dot_precision
      value: 52.851711026615966
    - type: dot_recall
      value: 76.62624035281146
    - type: euclidean_accuracy
      value: 67.7
    - type: euclidean_ap
      value: 69.70320170421651
    - type: euclidean_f1
      value: 62.55625562556255
    - type: euclidean_precision
      value: 52.851711026615966
    - type: euclidean_recall
      value: 76.62624035281146
    - type: manhattan_accuracy
      value: 67.75
    - type: manhattan_ap
      value: 69.67833816050764
    - type: manhattan_f1
      value: 62.734082397003746
    - type: manhattan_precision
      value: 54.515866558177386
    - type: manhattan_recall
      value: 73.8699007717751
    - type: max_accuracy
      value: 67.75
    - type: max_ap
      value: 69.70320170421651
    - type: max_f1
      value: 62.734082397003746
  - task:
      type: PairClassification
    dataset:
      type: paws-x
      name: MTEB PawsX (fr)
      config: fr
      split: test
      revision: 8a04d940a42cd40658986fdd8e3da561533a3646
    metrics:
    - type: cos_sim_accuracy
      value: 69.0
    - type: cos_sim_ap
      value: 71.36406639969131
    - type: cos_sim_f1
      value: 64.45993031358886
    - type: cos_sim_precision
      value: 53.12275664034458
    - type: cos_sim_recall
      value: 81.94905869324474
    - type: dot_accuracy
      value: 69.0
    - type: dot_ap
      value: 71.2599779415656
    - type: dot_f1
      value: 64.45993031358886
    - type: dot_precision
      value: 53.12275664034458
    - type: dot_recall
      value: 81.94905869324474
    - type: euclidean_accuracy
      value: 69.0
    - type: euclidean_ap
      value: 71.3126257271965
    - type: euclidean_f1
      value: 64.45993031358886
    - type: euclidean_precision
      value: 53.12275664034458
    - type: euclidean_recall
      value: 81.94905869324474
    - type: manhattan_accuracy
      value: 69.0
    - type: manhattan_ap
      value: 71.29361764028188
    - type: manhattan_f1
      value: 64.54789615040288
    - type: manhattan_precision
      value: 54.16979714500376
    - type: manhattan_recall
      value: 79.84496124031007
    - type: max_accuracy
      value: 69.0
    - type: max_ap
      value: 71.36406639969131
    - type: max_f1
      value: 64.54789615040288
  - task:
      type: PairClassification
    dataset:
      type: paws-x
      name: MTEB PawsX (ja)
      config: ja
      split: test
      revision: 8a04d940a42cd40658986fdd8e3da561533a3646
    metrics:
    - type: cos_sim_accuracy
      value: 63.849999999999994
    - type: cos_sim_ap
      value: 60.914955950361026
    - type: cos_sim_f1
      value: 62.4556422995032
    - type: cos_sim_precision
      value: 45.47803617571059
    - type: cos_sim_recall
      value: 99.66024915062289
    - type: dot_accuracy
      value: 63.849999999999994
    - type: dot_ap
      value: 60.808056565465506
    - type: dot_f1
      value: 62.4556422995032
    - type: dot_precision
      value: 45.47803617571059
    - type: dot_recall
      value: 99.66024915062289
    - type: euclidean_accuracy
      value: 63.849999999999994
    - type: euclidean_ap
      value: 60.8231492677072
    - type: euclidean_f1
      value: 62.4556422995032
    - type: euclidean_precision
      value: 45.47803617571059
    - type: euclidean_recall
      value: 99.66024915062289
    - type: manhattan_accuracy
      value: 63.800000000000004
    - type: manhattan_ap
      value: 60.86392751846975
    - type: manhattan_f1
      value: 62.43348705214614
    - type: manhattan_precision
      value: 45.45454545454545
    - type: manhattan_recall
      value: 99.66024915062289
    - type: max_accuracy
      value: 63.849999999999994
    - type: max_ap
      value: 60.914955950361026
    - type: max_f1
      value: 62.4556422995032
  - task:
      type: PairClassification
    dataset:
      type: paws-x
      name: MTEB PawsX (ko)
      config: ko
      split: test
      revision: 8a04d940a42cd40658986fdd8e3da561533a3646
    metrics:
    - type: cos_sim_accuracy
      value: 61.1
    - type: cos_sim_ap
      value: 58.40339411735916
    - type: cos_sim_f1
      value: 62.7906976744186
    - type: cos_sim_precision
      value: 46.55172413793103
    - type: cos_sim_recall
      value: 96.42857142857143
    - type: dot_accuracy
      value: 61.1
    - type: dot_ap
      value: 58.439189685586456
    - type: dot_f1
      value: 62.7906976744186
    - type: dot_precision
      value: 46.55172413793103
    - type: dot_recall
      value: 96.42857142857143
    - type: euclidean_accuracy
      value: 61.1
    - type: euclidean_ap
      value: 58.34968788203145
    - type: euclidean_f1
      value: 62.7906976744186
    - type: euclidean_precision
      value: 46.55172413793103
    - type: euclidean_recall
      value: 96.42857142857143
    - type: manhattan_accuracy
      value: 61.1
    - type: manhattan_ap
      value: 58.31504446861402
    - type: manhattan_f1
      value: 62.636562272396226
    - type: manhattan_precision
      value: 46.48648648648649
    - type: manhattan_recall
      value: 95.98214285714286
    - type: max_accuracy
      value: 61.1
    - type: max_ap
      value: 58.439189685586456
    - type: max_f1
      value: 62.7906976744186
  - task:
      type: PairClassification
    dataset:
      type: paws-x
      name: MTEB PawsX (zh)
      config: zh
      split: test
      revision: 8a04d940a42cd40658986fdd8e3da561533a3646
    metrics:
    - type: cos_sim_accuracy
      value: 64.2
    - type: cos_sim_ap
      value: 63.73722153283802
    - type: cos_sim_f1
      value: 62.52707581227437
    - type: cos_sim_precision
      value: 46.16204690831556
    - type: cos_sim_recall
      value: 96.86800894854586
    - type: dot_accuracy
      value: 64.2
    - type: dot_ap
      value: 63.67335241021108
    - type: dot_f1
      value: 62.52707581227437
    - type: dot_precision
      value: 46.16204690831556
    - type: dot_recall
      value: 96.86800894854586
    - type: euclidean_accuracy
      value: 64.2
    - type: euclidean_ap
      value: 63.77399571117368
    - type: euclidean_f1
      value: 62.52707581227437
    - type: euclidean_precision
      value: 46.16204690831556
    - type: euclidean_recall
      value: 96.86800894854586
    - type: manhattan_accuracy
      value: 64.5
    - type: manhattan_ap
      value: 63.747406783360816
    - type: manhattan_f1
      value: 62.58601955813112
    - type: manhattan_precision
      value: 46.27745045527584
    - type: manhattan_recall
      value: 96.64429530201343
    - type: max_accuracy
      value: 64.5
    - type: max_ap
      value: 63.77399571117368
    - type: max_f1
      value: 62.58601955813112
  - task:
      type: Classification
    dataset:
      type: PL-MTEB/polemo2_in
      name: MTEB PolEmo2.0-IN
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 52.797783933518005
    - type: f1
      value: 53.84971294048786
  - task:
      type: Classification
    dataset:
      type: PL-MTEB/polemo2_out
      name: MTEB PolEmo2.0-OUT
      config: default
      split: test
      revision: None
    metrics:
    - type: accuracy
      value: 38.40080971659919
    - type: f1
      value: 30.38990873840624
  - task:
      type: STS
    dataset:
      type: C-MTEB/QBQTC
      name: MTEB QBQTC
      config: default
      split: test
      revision: 790b0510dc52b1553e8c49f3d2afb48c0e5c48b7
    metrics:
    - type: cos_sim_pearson
      value: 23.34232568997104
    - type: cos_sim_spearman
      value: 24.47961936211083
    - type: euclidean_pearson
      value: 22.03140944610336
    - type: euclidean_spearman
      value: 24.47949166265398
    - type: manhattan_pearson
      value: 25.542406448726908
    - type: manhattan_spearman
      value: 28.655724283839533
  - task:
      type: Retrieval
    dataset:
      type: quora-pl
      name: MTEB Quora-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 59.938
    - type: map_at_10
      value: 72.734
    - type: map_at_100
      value: 73.564
    - type: map_at_1000
      value: 73.602
    - type: map_at_3
      value: 69.707
    - type: map_at_5
      value: 71.515
    - type: mrr_at_1
      value: 69.28
    - type: mrr_at_10
      value: 76.97500000000001
    - type: mrr_at_100
      value: 77.27199999999999
    - type: mrr_at_1000
      value: 77.28
    - type: mrr_at_3
      value: 75.355
    - type: mrr_at_5
      value: 76.389
    - type: ndcg_at_1
      value: 69.33
    - type: ndcg_at_10
      value: 77.61099999999999
    - type: ndcg_at_100
      value: 80.02
    - type: ndcg_at_1000
      value: 80.487
    - type: ndcg_at_3
      value: 73.764
    - type: ndcg_at_5
      value: 75.723
    - type: precision_at_1
      value: 69.33
    - type: precision_at_10
      value: 11.917
    - type: precision_at_100
      value: 1.447
    - type: precision_at_1000
      value: 0.154
    - type: precision_at_3
      value: 32.29
    - type: precision_at_5
      value: 21.432000000000002
    - type: recall_at_1
      value: 59.938
    - type: recall_at_10
      value: 87.252
    - type: recall_at_100
      value: 96.612
    - type: recall_at_1000
      value: 99.388
    - type: recall_at_3
      value: 76.264
    - type: recall_at_5
      value: 81.71000000000001
  - task:
      type: Retrieval
    dataset:
      type: quora
      name: MTEB QuoraRetrieval
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 61.458999999999996
    - type: map_at_10
      value: 73.90299999999999
    - type: map_at_100
      value: 74.733
    - type: map_at_1000
      value: 74.771
    - type: map_at_3
      value: 70.999
    - type: map_at_5
      value: 72.745
    - type: mrr_at_1
      value: 70.93
    - type: mrr_at_10
      value: 78.353
    - type: mrr_at_100
      value: 78.636
    - type: mrr_at_1000
      value: 78.644
    - type: mrr_at_3
      value: 76.908
    - type: mrr_at_5
      value: 77.807
    - type: ndcg_at_1
      value: 70.93
    - type: ndcg_at_10
      value: 78.625
    - type: ndcg_at_100
      value: 81.01
    - type: ndcg_at_1000
      value: 81.45700000000001
    - type: ndcg_at_3
      value: 75.045
    - type: ndcg_at_5
      value: 76.84299999999999
    - type: precision_at_1
      value: 70.93
    - type: precision_at_10
      value: 11.953
    - type: precision_at_100
      value: 1.4489999999999998
    - type: precision_at_1000
      value: 0.154
    - type: precision_at_3
      value: 32.65
    - type: precision_at_5
      value: 21.598
    - type: recall_at_1
      value: 61.458999999999996
    - type: recall_at_10
      value: 87.608
    - type: recall_at_100
      value: 96.818
    - type: recall_at_1000
      value: 99.445
    - type: recall_at_3
      value: 77.354
    - type: recall_at_5
      value: 82.334
  - task:
      type: Clustering
    dataset:
      type: mteb/reddit-clustering
      name: MTEB RedditClustering
      config: default
      split: test
      revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
    metrics:
    - type: v_measure
      value: 28.519889100999958
  - task:
      type: Clustering
    dataset:
      type: mteb/reddit-clustering-p2p
      name: MTEB RedditClusteringP2P
      config: default
      split: test
      revision: 282350215ef01743dc01b456c7f5241fa8937f16
    metrics:
    - type: v_measure
      value: 38.62765374782771
  - task:
      type: Retrieval
    dataset:
      type: scidocs
      name: MTEB SCIDOCS
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.52
    - type: map_at_10
      value: 0.893
    - type: map_at_100
      value: 1.113
    - type: map_at_1000
      value: 1.304
    - type: map_at_3
      value: 0.7779999999999999
    - type: map_at_5
      value: 0.8200000000000001
    - type: mrr_at_1
      value: 2.6
    - type: mrr_at_10
      value: 4.0680000000000005
    - type: mrr_at_100
      value: 4.6080000000000005
    - type: mrr_at_1000
      value: 4.797
    - type: mrr_at_3
      value: 3.5999999999999996
    - type: mrr_at_5
      value: 3.8150000000000004
    - type: ndcg_at_1
      value: 2.6
    - type: ndcg_at_10
      value: 1.79
    - type: ndcg_at_100
      value: 3.5549999999999997
    - type: ndcg_at_1000
      value: 9.942
    - type: ndcg_at_3
      value: 1.94
    - type: ndcg_at_5
      value: 1.543
    - type: precision_at_1
      value: 2.6
    - type: precision_at_10
      value: 0.8500000000000001
    - type: precision_at_100
      value: 0.361
    - type: precision_at_1000
      value: 0.197
    - type: precision_at_3
      value: 1.7670000000000001
    - type: precision_at_5
      value: 1.26
    - type: recall_at_1
      value: 0.52
    - type: recall_at_10
      value: 1.7149999999999999
    - type: recall_at_100
      value: 7.318
    - type: recall_at_1000
      value: 39.915
    - type: recall_at_3
      value: 1.0699999999999998
    - type: recall_at_5
      value: 1.27
  - task:
      type: Retrieval
    dataset:
      type: scidocs-pl
      name: MTEB SCIDOCS-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.32
    - type: map_at_10
      value: 0.676
    - type: map_at_100
      value: 0.847
    - type: map_at_1000
      value: 1.032
    - type: map_at_3
      value: 0.5369999999999999
    - type: map_at_5
      value: 0.592
    - type: mrr_at_1
      value: 1.6
    - type: mrr_at_10
      value: 2.863
    - type: mrr_at_100
      value: 3.334
    - type: mrr_at_1000
      value: 3.5479999999999996
    - type: mrr_at_3
      value: 2.317
    - type: mrr_at_5
      value: 2.587
    - type: ndcg_at_1
      value: 1.6
    - type: ndcg_at_10
      value: 1.397
    - type: ndcg_at_100
      value: 2.819
    - type: ndcg_at_1000
      value: 9.349
    - type: ndcg_at_3
      value: 1.3
    - type: ndcg_at_5
      value: 1.1079999999999999
    - type: precision_at_1
      value: 1.6
    - type: precision_at_10
      value: 0.74
    - type: precision_at_100
      value: 0.295
    - type: precision_at_1000
      value: 0.194
    - type: precision_at_3
      value: 1.2
    - type: precision_at_5
      value: 0.96
    - type: recall_at_1
      value: 0.32
    - type: recall_at_10
      value: 1.505
    - type: recall_at_100
      value: 5.988
    - type: recall_at_1000
      value: 39.308
    - type: recall_at_3
      value: 0.72
    - type: recall_at_5
      value: 0.9650000000000001
  - task:
      type: PairClassification
    dataset:
      type: PL-MTEB/sicke-pl-pairclassification
      name: MTEB SICK-E-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_accuracy
      value: 73.84834896045659
    - type: cos_sim_ap
      value: 55.484124732566606
    - type: cos_sim_f1
      value: 57.34228187919464
    - type: cos_sim_precision
      value: 46.01464885825076
    - type: cos_sim_recall
      value: 76.06837606837607
    - type: dot_accuracy
      value: 73.84834896045659
    - type: dot_ap
      value: 55.48400003295399
    - type: dot_f1
      value: 57.34228187919464
    - type: dot_precision
      value: 46.01464885825076
    - type: dot_recall
      value: 76.06837606837607
    - type: euclidean_accuracy
      value: 73.84834896045659
    - type: euclidean_ap
      value: 55.48407331902175
    - type: euclidean_f1
      value: 57.34228187919464
    - type: euclidean_precision
      value: 46.01464885825076
    - type: euclidean_recall
      value: 76.06837606837607
    - type: manhattan_accuracy
      value: 73.80758255197716
    - type: manhattan_ap
      value: 55.42477275597209
    - type: manhattan_f1
      value: 57.55860953920776
    - type: manhattan_precision
      value: 46.29388816644994
    - type: manhattan_recall
      value: 76.06837606837607
    - type: max_accuracy
      value: 73.84834896045659
    - type: max_ap
      value: 55.484124732566606
    - type: max_f1
      value: 57.55860953920776
  - task:
      type: STS
    dataset:
      type: mteb/sickr-sts
      name: MTEB SICK-R
      config: default
      split: test
      revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
    metrics:
    - type: cos_sim_pearson
      value: 67.03943120783973
    - type: cos_sim_spearman
      value: 62.93971145260584
    - type: euclidean_pearson
      value: 64.13947263916926
    - type: euclidean_spearman
      value: 62.93972324235839
    - type: manhattan_pearson
      value: 64.11295322654566
    - type: manhattan_spearman
      value: 62.92816122293202
  - task:
      type: STS
    dataset:
      type: PL-MTEB/sickr-pl-sts
      name: MTEB SICK-R-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: cos_sim_pearson
      value: 67.75034167381077
    - type: cos_sim_spearman
      value: 62.98158872758643
    - type: euclidean_pearson
      value: 64.25794794439082
    - type: euclidean_spearman
      value: 62.981566596223125
    - type: manhattan_pearson
      value: 64.25439446502435
    - type: manhattan_spearman
      value: 63.01301439900365
  - task:
      type: STS
    dataset:
      type: mteb/sts12-sts
      name: MTEB STS12
      config: default
      split: test
      revision: a0d554a64d88156834ff5ae9920b964011b16384
    metrics:
    - type: cos_sim_pearson
      value: 61.622204530882755
    - type: cos_sim_spearman
      value: 65.4632047656541
    - type: euclidean_pearson
      value: 59.21529585527598
    - type: euclidean_spearman
      value: 65.4638163967956
    - type: manhattan_pearson
      value: 59.39341472707122
    - type: manhattan_spearman
      value: 65.57635757250173
  - task:
      type: STS
    dataset:
      type: mteb/sts13-sts
      name: MTEB STS13
      config: default
      split: test
      revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
    metrics:
    - type: cos_sim_pearson
      value: 60.329743331971486
    - type: cos_sim_spearman
      value: 62.78607195958339
    - type: euclidean_pearson
      value: 62.07415212138581
    - type: euclidean_spearman
      value: 62.78618151904129
    - type: manhattan_pearson
      value: 62.41250554765521
    - type: manhattan_spearman
      value: 62.87580558029627
  - task:
      type: STS
    dataset:
      type: mteb/sts14-sts
      name: MTEB STS14
      config: default
      split: test
      revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
    metrics:
    - type: cos_sim_pearson
      value: 59.16277512775291
    - type: cos_sim_spearman
      value: 57.53693422381856
    - type: euclidean_pearson
      value: 57.85017283427473
    - type: euclidean_spearman
      value: 57.53697385589326
    - type: manhattan_pearson
      value: 58.049796184955596
    - type: manhattan_spearman
      value: 57.76174789162225
  - task:
      type: STS
    dataset:
      type: mteb/sts15-sts
      name: MTEB STS15
      config: default
      split: test
      revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
    metrics:
    - type: cos_sim_pearson
      value: 74.42588553600197
    - type: cos_sim_spearman
      value: 74.25087788257943
    - type: euclidean_pearson
      value: 73.35436018935222
    - type: euclidean_spearman
      value: 74.25087694991477
    - type: manhattan_pearson
      value: 73.33747415771185
    - type: manhattan_spearman
      value: 74.21504509447377
  - task:
      type: STS
    dataset:
      type: mteb/sts16-sts
      name: MTEB STS16
      config: default
      split: test
      revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
    metrics:
    - type: cos_sim_pearson
      value: 75.77242432372144
    - type: cos_sim_spearman
      value: 75.72930700521489
    - type: euclidean_pearson
      value: 75.6995220623788
    - type: euclidean_spearman
      value: 75.72930646047212
    - type: manhattan_pearson
      value: 75.65841087952896
    - type: manhattan_spearman
      value: 75.69567692328437
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (ko-ko)
      config: ko-ko
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 66.2495297342053
    - type: cos_sim_spearman
      value: 66.14124319602982
    - type: euclidean_pearson
      value: 66.49498096178358
    - type: euclidean_spearman
      value: 66.14121792287747
    - type: manhattan_pearson
      value: 66.51560623835172
    - type: manhattan_spearman
      value: 66.05794413582558
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (ar-ar)
      config: ar-ar
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 75.0045186560239
    - type: cos_sim_spearman
      value: 74.96504390762252
    - type: euclidean_pearson
      value: 74.20988464347049
    - type: euclidean_spearman
      value: 74.98114602301776
    - type: manhattan_pearson
      value: 74.37929169860529
    - type: manhattan_spearman
      value: 75.37049827509504
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (en-ar)
      config: en-ar
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 73.88478151514396
    - type: cos_sim_spearman
      value: 74.05322141272103
    - type: euclidean_pearson
      value: 73.52175483343693
    - type: euclidean_spearman
      value: 74.05322141272103
    - type: manhattan_pearson
      value: 73.35875118828287
    - type: manhattan_spearman
      value: 73.83972625384673
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (en-de)
      config: en-de
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 75.57014781622605
    - type: cos_sim_spearman
      value: 74.95329129562734
    - type: euclidean_pearson
      value: 75.5667786729257
    - type: euclidean_spearman
      value: 74.95329129562734
    - type: manhattan_pearson
      value: 75.39548673816147
    - type: manhattan_spearman
      value: 74.89428642503749
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (en-en)
      config: en-en
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 80.04007129652777
    - type: cos_sim_spearman
      value: 79.94429611477106
    - type: euclidean_pearson
      value: 79.91583070858822
    - type: euclidean_spearman
      value: 79.94429611477106
    - type: manhattan_pearson
      value: 80.14382273152769
    - type: manhattan_spearman
      value: 80.23862855392836
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (en-tr)
      config: en-tr
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 77.28740870194635
    - type: cos_sim_spearman
      value: 77.18286391819586
    - type: euclidean_pearson
      value: 77.05644328687119
    - type: euclidean_spearman
      value: 77.18286391819586
    - type: manhattan_pearson
      value: 77.15625898067294
    - type: manhattan_spearman
      value: 77.03165154316278
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (es-en)
      config: es-en
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 72.99293002371301
    - type: cos_sim_spearman
      value: 72.24657859872468
    - type: euclidean_pearson
      value: 73.38839879755461
    - type: euclidean_spearman
      value: 72.24657859872468
    - type: manhattan_pearson
      value: 73.6627728800822
    - type: manhattan_spearman
      value: 72.70893449698669
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (es-es)
      config: es-es
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 81.37213723705916
    - type: cos_sim_spearman
      value: 80.64548512701263
    - type: euclidean_pearson
      value: 80.94992193351284
    - type: euclidean_spearman
      value: 80.64484963200427
    - type: manhattan_pearson
      value: 80.92246813841794
    - type: manhattan_spearman
      value: 80.68860823161657
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (fr-en)
      config: fr-en
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 77.54059604962391
    - type: cos_sim_spearman
      value: 77.19559169700682
    - type: euclidean_pearson
      value: 77.32739821317861
    - type: euclidean_spearman
      value: 77.19559169700682
    - type: manhattan_pearson
      value: 77.29224328831437
    - type: manhattan_spearman
      value: 77.24394878313191
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (it-en)
      config: it-en
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 79.06397062195414
    - type: cos_sim_spearman
      value: 78.66694637555244
    - type: euclidean_pearson
      value: 79.34923290885872
    - type: euclidean_spearman
      value: 78.66694637555244
    - type: manhattan_pearson
      value: 79.50802161625809
    - type: manhattan_spearman
      value: 78.79195213396169
  - task:
      type: STS
    dataset:
      type: mteb/sts17-crosslingual-sts
      name: MTEB STS17 (nl-en)
      config: nl-en
      split: test
      revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
    metrics:
    - type: cos_sim_pearson
      value: 78.66045829245238
    - type: cos_sim_spearman
      value: 78.14055373851183
    - type: euclidean_pearson
      value: 78.94489279300518
    - type: euclidean_spearman
      value: 78.14055373851183
    - type: manhattan_pearson
      value: 79.33473165536323
    - type: manhattan_spearman
      value: 78.5783429705299
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (en)
      config: en
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 36.63454535818336
    - type: cos_sim_spearman
      value: 47.12016162570126
    - type: euclidean_pearson
      value: 39.07268779927362
    - type: euclidean_spearman
      value: 47.12016162570126
    - type: manhattan_pearson
      value: 41.723119770725944
    - type: manhattan_spearman
      value: 47.90334362422989
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (de)
      config: de
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 13.325547358617957
    - type: cos_sim_spearman
      value: 24.094051740693416
    - type: euclidean_pearson
      value: 10.39110006005262
    - type: euclidean_spearman
      value: 24.094051740693416
    - type: manhattan_pearson
      value: 12.4380555005162
    - type: manhattan_spearman
      value: 25.176800279885715
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (es)
      config: es
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 41.21281570342249
    - type: cos_sim_spearman
      value: 55.397885077207974
    - type: euclidean_pearson
      value: 43.96150945976646
    - type: euclidean_spearman
      value: 55.397885077207974
    - type: manhattan_pearson
      value: 49.58812224529121
    - type: manhattan_spearman
      value: 55.35874879475974
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (pl)
      config: pl
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 5.985012243744998
    - type: cos_sim_spearman
      value: 25.307464943919012
    - type: euclidean_pearson
      value: -4.080537702499046
    - type: euclidean_spearman
      value: 25.307464943919012
    - type: manhattan_pearson
      value: -2.5058642304196543
    - type: manhattan_spearman
      value: 26.751588484373233
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (tr)
      config: tr
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 34.44666578772084
    - type: cos_sim_spearman
      value: 46.45977141800899
    - type: euclidean_pearson
      value: 38.78305544036559
    - type: euclidean_spearman
      value: 46.45977141800899
    - type: manhattan_pearson
      value: 46.45101297876112
    - type: manhattan_spearman
      value: 50.642972694093814
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (ar)
      config: ar
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 28.095327083873055
    - type: cos_sim_spearman
      value: 40.24741745875892
    - type: euclidean_pearson
      value: 29.141496784653892
    - type: euclidean_spearman
      value: 40.24741745875892
    - type: manhattan_pearson
      value: 32.013290716034064
    - type: manhattan_spearman
      value: 40.85454084311211
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (ru)
      config: ru
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 27.46788309503312
    - type: cos_sim_spearman
      value: 43.57385391855994
    - type: euclidean_pearson
      value: 24.558349674326177
    - type: euclidean_spearman
      value: 43.57385391855994
    - type: manhattan_pearson
      value: 28.974505207055866
    - type: manhattan_spearman
      value: 44.111553205713
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (zh)
      config: zh
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 34.87841073990563
    - type: cos_sim_spearman
      value: 52.8221686505807
    - type: euclidean_pearson
      value: 38.36114580544504
    - type: euclidean_spearman
      value: 52.8221686505807
    - type: manhattan_pearson
      value: 46.69329448756753
    - type: manhattan_spearman
      value: 53.9140781097337
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (fr)
      config: fr
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 49.999267528357
    - type: cos_sim_spearman
      value: 61.71837669697145
    - type: euclidean_pearson
      value: 53.578476744372274
    - type: euclidean_spearman
      value: 61.71837669697145
    - type: manhattan_pearson
      value: 56.410294227490795
    - type: manhattan_spearman
      value: 60.684457655864875
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (de-en)
      config: de-en
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 22.43564137760586
    - type: cos_sim_spearman
      value: 34.28346144104183
    - type: euclidean_pearson
      value: 27.41326011184764
    - type: euclidean_spearman
      value: 34.28346144104183
    - type: manhattan_pearson
      value: 35.62923154232163
    - type: manhattan_spearman
      value: 37.937151135297185
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (es-en)
      config: es-en
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 44.34071611983998
    - type: cos_sim_spearman
      value: 57.823185616169646
    - type: euclidean_pearson
      value: 49.29310650157244
    - type: euclidean_spearman
      value: 57.823185616169646
    - type: manhattan_pearson
      value: 55.93298736518848
    - type: manhattan_spearman
      value: 58.57556581684834
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (it)
      config: it
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 56.07027840344927
    - type: cos_sim_spearman
      value: 62.20158260763411
    - type: euclidean_pearson
      value: 55.887969718543616
    - type: euclidean_spearman
      value: 62.20158260763411
    - type: manhattan_pearson
      value: 56.081533365738444
    - type: manhattan_spearman
      value: 62.018651361750685
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (pl-en)
      config: pl-en
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 41.41816324477061
    - type: cos_sim_spearman
      value: 44.71684955996943
    - type: euclidean_pearson
      value: 42.74585025834968
    - type: euclidean_spearman
      value: 44.71684955996943
    - type: manhattan_pearson
      value: 47.992481632815256
    - type: manhattan_spearman
      value: 46.18587933349126
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (zh-en)
      config: zh-en
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 38.89140730917917
    - type: cos_sim_spearman
      value: 49.18633779347391
    - type: euclidean_pearson
      value: 43.27605428753535
    - type: euclidean_spearman
      value: 49.18633779347391
    - type: manhattan_pearson
      value: 48.22046568809415
    - type: manhattan_spearman
      value: 49.248416391249464
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (es-it)
      config: es-it
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 40.31620568726327
    - type: cos_sim_spearman
      value: 49.13034440774138
    - type: euclidean_pearson
      value: 43.95169508285692
    - type: euclidean_spearman
      value: 49.13034440774138
    - type: manhattan_pearson
      value: 48.84250981398146
    - type: manhattan_spearman
      value: 49.54216339903405
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (de-fr)
      config: de-fr
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 27.074582378144058
    - type: cos_sim_spearman
      value: 41.29498619968451
    - type: euclidean_pearson
      value: 28.993986097276505
    - type: euclidean_spearman
      value: 41.29498619968451
    - type: manhattan_pearson
      value: 32.079813951133254
    - type: manhattan_spearman
      value: 43.664111732941464
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (de-pl)
      config: de-pl
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 6.864334110072116
    - type: cos_sim_spearman
      value: 25.805458732687914
    - type: euclidean_pearson
      value: 11.435920047618103
    - type: euclidean_spearman
      value: 25.805458732687914
    - type: manhattan_pearson
      value: 15.036308569899552
    - type: manhattan_spearman
      value: 25.405135387192757
  - task:
      type: STS
    dataset:
      type: mteb/sts22-crosslingual-sts
      name: MTEB STS22 (fr-pl)
      config: fr-pl
      split: test
      revision: eea2b4fe26a775864c896887d910b76a8098ad3f
    metrics:
    - type: cos_sim_pearson
      value: 65.44029549925597
    - type: cos_sim_spearman
      value: 61.97797868009122
    - type: euclidean_pearson
      value: 65.92740669959876
    - type: euclidean_spearman
      value: 61.97797868009122
    - type: manhattan_pearson
      value: 70.29575044091207
    - type: manhattan_spearman
      value: 73.24670207647144
  - task:
      type: STS
    dataset:
      type: C-MTEB/STSB
      name: MTEB STSB
      config: default
      split: test
      revision: 0cde68302b3541bb8b3c340dc0644b0b745b3dc0
    metrics:
    - type: cos_sim_pearson
      value: 51.35413149349556
    - type: cos_sim_spearman
      value: 50.175051356729924
    - type: euclidean_pearson
      value: 53.12039152785364
    - type: euclidean_spearman
      value: 50.174289111089685
    - type: manhattan_pearson
      value: 53.0731746793555
    - type: manhattan_spearman
      value: 50.15176393928403
  - task:
      type: STS
    dataset:
      type: mteb/stsbenchmark-sts
      name: MTEB STSBenchmark
      config: default
      split: test
      revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
    metrics:
    - type: cos_sim_pearson
      value: 67.84222983023291
    - type: cos_sim_spearman
      value: 67.39086924655895
    - type: euclidean_pearson
      value: 67.3393327127967
    - type: euclidean_spearman
      value: 67.39088047106472
    - type: manhattan_pearson
      value: 67.40316731822271
    - type: manhattan_spearman
      value: 67.49067800994015
  - task:
      type: Classification
    dataset:
      type: ScandEval/scala-da
      name: MTEB ScalaDaClassification
      config: default
      split: test
      revision: 1de08520a7b361e92ffa2a2201ebd41942c54675
    metrics:
    - type: accuracy
      value: 50.62988281250001
    - type: ap
      value: 50.32274824114816
    - type: f1
      value: 50.37741703766756
  - task:
      type: Classification
    dataset:
      type: ScandEval/scala-nb
      name: MTEB ScalaNbClassification
      config: default
      split: test
      revision: 237111a078ad5a834a55c57803d40bbe410ed03b
    metrics:
    - type: accuracy
      value: 51.181640625
    - type: ap
      value: 50.60884394099696
    - type: f1
      value: 50.866988720930415
  - task:
      type: Classification
    dataset:
      type: ScandEval/scala-nn
      name: MTEB ScalaNnClassification
      config: default
      split: test
      revision: 9d9a2a4092ed3cacf0744592f6d2f32ab8ef4c0b
    metrics:
    - type: accuracy
      value: 50.9375
    - type: ap
      value: 50.47969135089731
    - type: f1
      value: 50.62913552324756
  - task:
      type: Classification
    dataset:
      type: ScandEval/scala-sv
      name: MTEB ScalaSvClassification
      config: default
      split: test
      revision: 1b48e3dcb02872335ff985ff938a054a4ed99008
    metrics:
    - type: accuracy
      value: 51.1474609375
    - type: ap
      value: 50.5894187272385
    - type: f1
      value: 50.901812392367916
  - task:
      type: Reranking
    dataset:
      type: mteb/scidocs-reranking
      name: MTEB SciDocsRR
      config: default
      split: test
      revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
    metrics:
    - type: map
      value: 68.36051662289248
    - type: mrr
      value: 89.39224265204656
  - task:
      type: Retrieval
    dataset:
      type: scifact
      name: MTEB SciFact
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 23.721999999999998
    - type: map_at_10
      value: 31.335
    - type: map_at_100
      value: 32.461
    - type: map_at_1000
      value: 32.557
    - type: map_at_3
      value: 29.282000000000004
    - type: map_at_5
      value: 30.602
    - type: mrr_at_1
      value: 24.667
    - type: mrr_at_10
      value: 32.363
    - type: mrr_at_100
      value: 33.421
    - type: mrr_at_1000
      value: 33.499
    - type: mrr_at_3
      value: 30.444
    - type: mrr_at_5
      value: 31.628
    - type: ndcg_at_1
      value: 24.667
    - type: ndcg_at_10
      value: 35.29
    - type: ndcg_at_100
      value: 40.665
    - type: ndcg_at_1000
      value: 43.241
    - type: ndcg_at_3
      value: 31.238
    - type: ndcg_at_5
      value: 33.486
    - type: precision_at_1
      value: 24.667
    - type: precision_at_10
      value: 5.1
    - type: precision_at_100
      value: 0.7969999999999999
    - type: precision_at_1000
      value: 0.10300000000000001
    - type: precision_at_3
      value: 12.667
    - type: precision_at_5
      value: 8.933
    - type: recall_at_1
      value: 23.721999999999998
    - type: recall_at_10
      value: 46.417
    - type: recall_at_100
      value: 70.944
    - type: recall_at_1000
      value: 91.033
    - type: recall_at_3
      value: 35.693999999999996
    - type: recall_at_5
      value: 40.944
  - task:
      type: Retrieval
    dataset:
      type: scifact-pl
      name: MTEB SciFact-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 21.706
    - type: map_at_10
      value: 28.333000000000002
    - type: map_at_100
      value: 29.364
    - type: map_at_1000
      value: 29.451
    - type: map_at_3
      value: 26.112999999999996
    - type: map_at_5
      value: 27.502
    - type: mrr_at_1
      value: 23.0
    - type: mrr_at_10
      value: 29.555999999999997
    - type: mrr_at_100
      value: 30.536
    - type: mrr_at_1000
      value: 30.606
    - type: mrr_at_3
      value: 27.333000000000002
    - type: mrr_at_5
      value: 28.717
    - type: ndcg_at_1
      value: 23.0
    - type: ndcg_at_10
      value: 32.238
    - type: ndcg_at_100
      value: 37.785999999999994
    - type: ndcg_at_1000
      value: 40.266999999999996
    - type: ndcg_at_3
      value: 27.961000000000002
    - type: ndcg_at_5
      value: 30.322
    - type: precision_at_1
      value: 23.0
    - type: precision_at_10
      value: 4.7669999999999995
    - type: precision_at_100
      value: 0.787
    - type: precision_at_1000
      value: 0.10200000000000001
    - type: precision_at_3
      value: 11.444
    - type: precision_at_5
      value: 8.200000000000001
    - type: recall_at_1
      value: 21.706
    - type: recall_at_10
      value: 43.206
    - type: recall_at_100
      value: 69.678
    - type: recall_at_1000
      value: 89.333
    - type: recall_at_3
      value: 31.900000000000002
    - type: recall_at_5
      value: 37.594
  - task:
      type: PairClassification
    dataset:
      type: mteb/sprintduplicatequestions-pairclassification
      name: MTEB SprintDuplicateQuestions
      config: default
      split: test
      revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
    metrics:
    - type: cos_sim_accuracy
      value: 99.5
    - type: cos_sim_ap
      value: 77.07584309978081
    - type: cos_sim_f1
      value: 71.8864950078823
    - type: cos_sim_precision
      value: 75.74750830564784
    - type: cos_sim_recall
      value: 68.4
    - type: dot_accuracy
      value: 99.5
    - type: dot_ap
      value: 77.07584309978081
    - type: dot_f1
      value: 71.8864950078823
    - type: dot_precision
      value: 75.74750830564784
    - type: dot_recall
      value: 68.4
    - type: euclidean_accuracy
      value: 99.5
    - type: euclidean_ap
      value: 77.07584309978081
    - type: euclidean_f1
      value: 71.8864950078823
    - type: euclidean_precision
      value: 75.74750830564784
    - type: euclidean_recall
      value: 68.4
    - type: manhattan_accuracy
      value: 99.50594059405941
    - type: manhattan_ap
      value: 77.41658577240027
    - type: manhattan_f1
      value: 71.91374663072777
    - type: manhattan_precision
      value: 78.01169590643275
    - type: manhattan_recall
      value: 66.7
    - type: max_accuracy
      value: 99.50594059405941
    - type: max_ap
      value: 77.41658577240027
    - type: max_f1
      value: 71.91374663072777
  - task:
      type: Clustering
    dataset:
      type: mteb/stackexchange-clustering
      name: MTEB StackExchangeClustering
      config: default
      split: test
      revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
    metrics:
    - type: v_measure
      value: 46.32521494308228
  - task:
      type: Clustering
    dataset:
      type: mteb/stackexchange-clustering-p2p
      name: MTEB StackExchangeClusteringP2P
      config: default
      split: test
      revision: 815ca46b2622cec33ccafc3735d572c266efdb44
    metrics:
    - type: v_measure
      value: 20.573273825125266
  - task:
      type: Reranking
    dataset:
      type: mteb/stackoverflowdupquestions-reranking
      name: MTEB StackOverflowDupQuestions
      config: default
      split: test
      revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
    metrics:
    - type: map
      value: 38.612724125942385
    - type: mrr
      value: 38.891130315762666
  - task:
      type: Summarization
    dataset:
      type: mteb/summeval
      name: MTEB SummEval
      config: default
      split: test
      revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
    metrics:
    - type: cos_sim_pearson
      value: 29.305330424238836
    - type: cos_sim_spearman
      value: 30.556621737388685
    - type: dot_pearson
      value: 29.30533056265583
    - type: dot_spearman
      value: 30.556621737388685
  - task:
      type: Classification
    dataset:
      type: ScandEval/swerec-mini
      name: MTEB SweRecClassification
      config: default
      split: test
      revision: 3c62f26bafdc4c4e1c16401ad4b32f0a94b46612
    metrics:
    - type: accuracy
      value: 68.4716796875
    - type: f1
      value: 59.865730786092364
  - task:
      type: Reranking
    dataset:
      type: C-MTEB/T2Reranking
      name: MTEB T2Reranking
      config: default
      split: dev
      revision: 76631901a18387f85eaa53e5450019b87ad58ef9
    metrics:
    - type: map
      value: 55.34794621490011
    - type: mrr
      value: 59.22764129348421
  - task:
      type: Retrieval
    dataset:
      type: C-MTEB/T2Retrieval
      name: MTEB T2Retrieval
      config: default
      split: dev
      revision: 8731a845f1bf500a4f111cf1070785c793d10e64
    metrics:
    - type: map_at_1
      value: 0.586
    - type: map_at_10
      value: 0.819
    - type: map_at_100
      value: 0.8920000000000001
    - type: map_at_1000
      value: 0.928
    - type: map_at_3
      value: 0.729
    - type: map_at_5
      value: 0.771
    - type: mrr_at_1
      value: 1.9949999999999999
    - type: mrr_at_10
      value: 2.608
    - type: mrr_at_100
      value: 2.771
    - type: mrr_at_1000
      value: 2.8289999999999997
    - type: mrr_at_3
      value: 2.365
    - type: mrr_at_5
      value: 2.483
    - type: ndcg_at_1
      value: 1.9949999999999999
    - type: ndcg_at_10
      value: 1.314
    - type: ndcg_at_100
      value: 1.831
    - type: ndcg_at_1000
      value: 3.4139999999999997
    - type: ndcg_at_3
      value: 1.377
    - type: ndcg_at_5
      value: 1.2630000000000001
    - type: precision_at_1
      value: 1.9949999999999999
    - type: precision_at_10
      value: 0.488
    - type: precision_at_100
      value: 0.123
    - type: precision_at_1000
      value: 0.054
    - type: precision_at_3
      value: 1.027
    - type: precision_at_5
      value: 0.737
    - type: recall_at_1
      value: 0.586
    - type: recall_at_10
      value: 1.3390000000000002
    - type: recall_at_100
      value: 3.15
    - type: recall_at_1000
      value: 11.859
    - type: recall_at_3
      value: 0.8710000000000001
    - type: recall_at_5
      value: 1.0290000000000001
  - task:
      type: Classification
    dataset:
      type: C-MTEB/TNews-classification
      name: MTEB TNews
      config: default
      split: validation
      revision: 317f262bf1e6126357bbe89e875451e4b0938fe4
    metrics:
    - type: accuracy
      value: 40.946
    - type: f1
      value: 39.56517169731474
  - task:
      type: Retrieval
    dataset:
      type: trec-covid
      name: MTEB TRECCOVID
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.08499999999999999
    - type: map_at_10
      value: 0.462
    - type: map_at_100
      value: 0.893
    - type: map_at_1000
      value: 1.129
    - type: map_at_3
      value: 0.232
    - type: map_at_5
      value: 0.3
    - type: mrr_at_1
      value: 38.0
    - type: mrr_at_10
      value: 50.629999999999995
    - type: mrr_at_100
      value: 51.315999999999995
    - type: mrr_at_1000
      value: 51.365
    - type: mrr_at_3
      value: 47.0
    - type: mrr_at_5
      value: 48.9
    - type: ndcg_at_1
      value: 31.0
    - type: ndcg_at_10
      value: 24.823
    - type: ndcg_at_100
      value: 10.583
    - type: ndcg_at_1000
      value: 6.497999999999999
    - type: ndcg_at_3
      value: 30.95
    - type: ndcg_at_5
      value: 27.899
    - type: precision_at_1
      value: 38.0
    - type: precision_at_10
      value: 25.6
    - type: precision_at_100
      value: 8.98
    - type: precision_at_1000
      value: 2.248
    - type: precision_at_3
      value: 34.666999999999994
    - type: precision_at_5
      value: 29.599999999999998
    - type: recall_at_1
      value: 0.08499999999999999
    - type: recall_at_10
      value: 0.641
    - type: recall_at_100
      value: 2.002
    - type: recall_at_1000
      value: 4.902
    - type: recall_at_3
      value: 0.28200000000000003
    - type: recall_at_5
      value: 0.379
  - task:
      type: Retrieval
    dataset:
      type: trec-covid-pl
      name: MTEB TRECCOVID-PL
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.124
    - type: map_at_10
      value: 0.45199999999999996
    - type: map_at_100
      value: 0.874
    - type: map_at_1000
      value: 1.1039999999999999
    - type: map_at_3
      value: 0.253
    - type: map_at_5
      value: 0.32299999999999995
    - type: mrr_at_1
      value: 36.0
    - type: mrr_at_10
      value: 47.56
    - type: mrr_at_100
      value: 48.532
    - type: mrr_at_1000
      value: 48.579
    - type: mrr_at_3
      value: 45.0
    - type: mrr_at_5
      value: 45.5
    - type: ndcg_at_1
      value: 34.0
    - type: ndcg_at_10
      value: 24.529
    - type: ndcg_at_100
      value: 10.427
    - type: ndcg_at_1000
      value: 6.457
    - type: ndcg_at_3
      value: 31.173000000000002
    - type: ndcg_at_5
      value: 27.738000000000003
    - type: precision_at_1
      value: 38.0
    - type: precision_at_10
      value: 25.4
    - type: precision_at_100
      value: 8.88
    - type: precision_at_1000
      value: 2.2159999999999997
    - type: precision_at_3
      value: 34.666999999999994
    - type: precision_at_5
      value: 29.2
    - type: recall_at_1
      value: 0.124
    - type: recall_at_10
      value: 0.618
    - type: recall_at_100
      value: 1.9349999999999998
    - type: recall_at_1000
      value: 4.808
    - type: recall_at_3
      value: 0.28300000000000003
    - type: recall_at_5
      value: 0.382
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (sqi-eng)
      config: sqi-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.9
    - type: f1
      value: 98.55000000000001
    - type: precision
      value: 98.38333333333334
    - type: recall
      value: 98.9
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (fry-eng)
      config: fry-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 65.3179190751445
    - type: f1
      value: 59.44582071749702
    - type: precision
      value: 57.49678869621066
    - type: recall
      value: 65.3179190751445
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (kur-eng)
      config: kur-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 38.53658536585366
    - type: f1
      value: 34.217555952803785
    - type: precision
      value: 32.96511296649355
    - type: recall
      value: 38.53658536585366
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (tur-eng)
      config: tur-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.7
    - type: f1
      value: 98.26666666666665
    - type: precision
      value: 98.05
    - type: recall
      value: 98.7
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (deu-eng)
      config: deu-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 99.3
    - type: f1
      value: 99.13333333333333
    - type: precision
      value: 99.05000000000001
    - type: recall
      value: 99.3
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (nld-eng)
      config: nld-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.89999999999999
    - type: f1
      value: 97.2
    - type: precision
      value: 96.85000000000001
    - type: recall
      value: 97.89999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ron-eng)
      config: ron-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.2
    - type: f1
      value: 97.6
    - type: precision
      value: 97.3
    - type: recall
      value: 98.2
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ang-eng)
      config: ang-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 52.23880597014925
    - type: f1
      value: 46.340992406389105
    - type: precision
      value: 44.556384742951906
    - type: recall
      value: 52.23880597014925
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ido-eng)
      config: ido-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.0
    - type: f1
      value: 93.67000000000002
    - type: precision
      value: 93.075
    - type: recall
      value: 95.0
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (jav-eng)
      config: jav-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 88.29268292682927
    - type: f1
      value: 85.76422764227642
    - type: precision
      value: 84.84204413472706
    - type: recall
      value: 88.29268292682927
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (isl-eng)
      config: isl-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.2
    - type: f1
      value: 96.46666666666667
    - type: precision
      value: 96.1
    - type: recall
      value: 97.2
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (slv-eng)
      config: slv-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.8408262454435
    - type: f1
      value: 95.9902794653706
    - type: precision
      value: 95.56500607533415
    - type: recall
      value: 96.8408262454435
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (cym-eng)
      config: cym-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 93.3913043478261
    - type: f1
      value: 91.30434782608695
    - type: precision
      value: 90.28985507246377
    - type: recall
      value: 93.3913043478261
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (kaz-eng)
      config: kaz-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 90.6086956521739
    - type: f1
      value: 88.1159420289855
    - type: precision
      value: 86.9623188405797
    - type: recall
      value: 90.6086956521739
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (est-eng)
      config: est-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.8
    - type: f1
      value: 97.16666666666667
    - type: precision
      value: 96.86666666666667
    - type: recall
      value: 97.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (heb-eng)
      config: heb-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 94.0
    - type: f1
      value: 92.34
    - type: precision
      value: 91.54166666666667
    - type: recall
      value: 94.0
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (gla-eng)
      config: gla-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 84.92159227985525
    - type: f1
      value: 80.8868975817106
    - type: precision
      value: 79.11540008041817
    - type: recall
      value: 84.92159227985525
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (mar-eng)
      config: mar-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 94.89999999999999
    - type: f1
      value: 93.35
    - type: precision
      value: 92.58333333333334
    - type: recall
      value: 94.89999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (lat-eng)
      config: lat-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 43.3
    - type: f1
      value: 36.64473116255726
    - type: precision
      value: 34.64017752457381
    - type: recall
      value: 43.3
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (bel-eng)
      config: bel-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.7
    - type: f1
      value: 95.68333333333332
    - type: precision
      value: 95.19999999999999
    - type: recall
      value: 96.7
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (pms-eng)
      config: pms-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 70.47619047619048
    - type: f1
      value: 66.63032734461306
    - type: precision
      value: 65.46459191863879
    - type: recall
      value: 70.47619047619048
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (gle-eng)
      config: gle-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 93.5
    - type: f1
      value: 91.63
    - type: precision
      value: 90.75
    - type: recall
      value: 93.5
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (pes-eng)
      config: pes-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.5
    - type: f1
      value: 94.36666666666666
    - type: precision
      value: 93.83333333333333
    - type: recall
      value: 95.5
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (nob-eng)
      config: nob-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 99.3
    - type: f1
      value: 99.06666666666666
    - type: precision
      value: 98.95
    - type: recall
      value: 99.3
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (bul-eng)
      config: bul-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.8
    - type: f1
      value: 94.51666666666667
    - type: precision
      value: 93.88333333333334
    - type: recall
      value: 95.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (cbk-eng)
      config: cbk-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 84.0
    - type: f1
      value: 80.46675324675326
    - type: precision
      value: 78.95999999999998
    - type: recall
      value: 84.0
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (hun-eng)
      config: hun-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.7
    - type: f1
      value: 96.93333333333332
    - type: precision
      value: 96.55
    - type: recall
      value: 97.7
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (uig-eng)
      config: uig-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 92.10000000000001
    - type: f1
      value: 90.07333333333334
    - type: precision
      value: 89.16166666666668
    - type: recall
      value: 92.10000000000001
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (rus-eng)
      config: rus-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.6
    - type: f1
      value: 94.35
    - type: precision
      value: 93.75
    - type: recall
      value: 95.6
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (spa-eng)
      config: spa-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.9
    - type: f1
      value: 98.53333333333335
    - type: precision
      value: 98.35000000000001
    - type: recall
      value: 98.9
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (hye-eng)
      config: hye-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.22641509433963
    - type: f1
      value: 95.14824797843666
    - type: precision
      value: 94.60916442048517
    - type: recall
      value: 96.22641509433963
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (tel-eng)
      config: tel-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 93.58974358974359
    - type: f1
      value: 91.59544159544159
    - type: precision
      value: 90.66951566951566
    - type: recall
      value: 93.58974358974359
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (afr-eng)
      config: afr-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.1
    - type: f1
      value: 97.46666666666668
    - type: precision
      value: 97.15
    - type: recall
      value: 98.1
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (mon-eng)
      config: mon-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 93.4090909090909
    - type: f1
      value: 91.5909090909091
    - type: precision
      value: 90.71969696969697
    - type: recall
      value: 93.4090909090909
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (arz-eng)
      config: arz-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 89.51781970649894
    - type: f1
      value: 86.76150544075072
    - type: precision
      value: 85.55206149545772
    - type: recall
      value: 89.51781970649894
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (hrv-eng)
      config: hrv-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.2
    - type: f1
      value: 97.65
    - type: precision
      value: 97.38333333333333
    - type: recall
      value: 98.2
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (nov-eng)
      config: nov-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 75.87548638132296
    - type: f1
      value: 71.24698906800073
    - type: precision
      value: 69.66572338167668
    - type: recall
      value: 75.87548638132296
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (gsw-eng)
      config: gsw-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 61.53846153846154
    - type: f1
      value: 54.83234714003944
    - type: precision
      value: 52.06552706552707
    - type: recall
      value: 61.53846153846154
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (nds-eng)
      config: nds-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 59.199999999999996
    - type: f1
      value: 54.183211233211225
    - type: precision
      value: 52.48751719986241
    - type: recall
      value: 59.199999999999996
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ukr-eng)
      config: ukr-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.6
    - type: f1
      value: 94.3
    - type: precision
      value: 93.65
    - type: recall
      value: 95.6
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (uzb-eng)
      config: uzb-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 87.85046728971963
    - type: f1
      value: 85.25700934579439
    - type: precision
      value: 84.09267912772586
    - type: recall
      value: 87.85046728971963
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (lit-eng)
      config: lit-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.0
    - type: f1
      value: 97.43333333333332
    - type: precision
      value: 97.15
    - type: recall
      value: 98.0
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ina-eng)
      config: ina-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 90.8
    - type: f1
      value: 88.66055555555555
    - type: precision
      value: 87.81845238095238
    - type: recall
      value: 90.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (lfn-eng)
      config: lfn-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 70.6
    - type: f1
      value: 65.538895353013
    - type: precision
      value: 63.69531394330308
    - type: recall
      value: 70.6
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (zsm-eng)
      config: zsm-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.89999999999999
    - type: f1
      value: 96.06666666666668
    - type: precision
      value: 95.68333333333334
    - type: recall
      value: 96.89999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ita-eng)
      config: ita-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.8
    - type: f1
      value: 95.95
    - type: precision
      value: 95.55
    - type: recall
      value: 96.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (cmn-eng)
      config: cmn-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.19999999999999
    - type: f1
      value: 93.8
    - type: precision
      value: 93.13333333333334
    - type: recall
      value: 95.19999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (lvs-eng)
      config: lvs-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.5
    - type: f1
      value: 95.45
    - type: precision
      value: 94.93333333333334
    - type: recall
      value: 96.5
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (glg-eng)
      config: glg-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.89999999999999
    - type: f1
      value: 97.28333333333332
    - type: precision
      value: 96.98333333333333
    - type: recall
      value: 97.89999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ceb-eng)
      config: ceb-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 78.16666666666666
    - type: f1
      value: 74.67336721249764
    - type: precision
      value: 73.26035353535354
    - type: recall
      value: 78.16666666666666
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (bre-eng)
      config: bre-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 11.200000000000001
    - type: f1
      value: 8.48123815073815
    - type: precision
      value: 7.843657708032708
    - type: recall
      value: 11.200000000000001
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ben-eng)
      config: ben-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 91.3
    - type: f1
      value: 89.02333333333333
    - type: precision
      value: 87.97500000000001
    - type: recall
      value: 91.3
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (swg-eng)
      config: swg-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 72.32142857142857
    - type: f1
      value: 67.69209956709956
    - type: precision
      value: 66.19047619047619
    - type: recall
      value: 72.32142857142857
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (arq-eng)
      config: arq-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 79.69264544456641
    - type: f1
      value: 75.40693115885212
    - type: precision
      value: 73.67544822539335
    - type: recall
      value: 79.69264544456641
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (kab-eng)
      config: kab-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 86.8
    - type: f1
      value: 83.65666666666667
    - type: precision
      value: 82.24833333333333
    - type: recall
      value: 86.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (fra-eng)
      config: fra-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.39999999999999
    - type: f1
      value: 95.36666666666666
    - type: precision
      value: 94.86666666666666
    - type: recall
      value: 96.39999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (por-eng)
      config: por-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.3
    - type: f1
      value: 95.49
    - type: precision
      value: 95.10833333333333
    - type: recall
      value: 96.3
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (tat-eng)
      config: tat-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 89.60000000000001
    - type: f1
      value: 87.04746031746032
    - type: precision
      value: 85.89583333333333
    - type: recall
      value: 89.60000000000001
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (oci-eng)
      config: oci-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 86.9
    - type: f1
      value: 84.57088023088022
    - type: precision
      value: 83.6475
    - type: recall
      value: 86.9
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (pol-eng)
      config: pol-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.2
    - type: f1
      value: 97.7
    - type: precision
      value: 97.46666666666668
    - type: recall
      value: 98.2
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (war-eng)
      config: war-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 85.39999999999999
    - type: f1
      value: 82.83333333333333
    - type: precision
      value: 81.80137426900586
    - type: recall
      value: 85.39999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (aze-eng)
      config: aze-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 91.4
    - type: f1
      value: 89.11999999999999
    - type: precision
      value: 88.12777777777778
    - type: recall
      value: 91.4
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (vie-eng)
      config: vie-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.8
    - type: f1
      value: 97.16666666666669
    - type: precision
      value: 96.85000000000001
    - type: recall
      value: 97.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (nno-eng)
      config: nno-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.89999999999999
    - type: f1
      value: 97.30666666666666
    - type: precision
      value: 97.02499999999999
    - type: recall
      value: 97.89999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (cha-eng)
      config: cha-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 27.00729927007299
    - type: f1
      value: 25.114895917815623
    - type: precision
      value: 24.602283361407448
    - type: recall
      value: 27.00729927007299
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (mhr-eng)
      config: mhr-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 14.099999999999998
    - type: f1
      value: 11.869284007509814
    - type: precision
      value: 11.199695454818405
    - type: recall
      value: 14.099999999999998
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (dan-eng)
      config: dan-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.7
    - type: f1
      value: 97.09
    - type: precision
      value: 96.80833333333332
    - type: recall
      value: 97.7
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ell-eng)
      config: ell-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.5
    - type: f1
      value: 95.47333333333333
    - type: precision
      value: 94.975
    - type: recall
      value: 96.5
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (amh-eng)
      config: amh-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 93.45238095238095
    - type: f1
      value: 91.66666666666666
    - type: precision
      value: 90.77380952380952
    - type: recall
      value: 93.45238095238095
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (pam-eng)
      config: pam-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 11.899999999999999
    - type: f1
      value: 10.303261315113037
    - type: precision
      value: 9.902986584515606
    - type: recall
      value: 11.899999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (hsb-eng)
      config: hsb-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 81.57349896480332
    - type: f1
      value: 77.86519438693352
    - type: precision
      value: 76.35595081247254
    - type: recall
      value: 81.57349896480332
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (srp-eng)
      config: srp-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.1
    - type: f1
      value: 94.86666666666667
    - type: precision
      value: 94.25
    - type: recall
      value: 96.1
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (epo-eng)
      config: epo-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.8
    - type: f1
      value: 98.46666666666667
    - type: precision
      value: 98.3
    - type: recall
      value: 98.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (kzj-eng)
      config: kzj-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 10.7
    - type: f1
      value: 8.621683883854935
    - type: precision
      value: 8.188292731521031
    - type: recall
      value: 10.7
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (awa-eng)
      config: awa-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 90.47619047619048
    - type: f1
      value: 87.8581735724593
    - type: precision
      value: 86.72438672438673
    - type: recall
      value: 90.47619047619048
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (fao-eng)
      config: fao-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.0381679389313
    - type: f1
      value: 93.60050890585242
    - type: precision
      value: 92.970737913486
    - type: recall
      value: 95.0381679389313
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (mal-eng)
      config: mal-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.2532751091703
    - type: f1
      value: 97.67103347889375
    - type: precision
      value: 97.37991266375546
    - type: recall
      value: 98.2532751091703
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ile-eng)
      config: ile-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 84.6
    - type: f1
      value: 80.99904761904763
    - type: precision
      value: 79.54634920634919
    - type: recall
      value: 84.6
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (bos-eng)
      config: bos-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.89265536723164
    - type: f1
      value: 95.90395480225989
    - type: precision
      value: 95.4331450094162
    - type: recall
      value: 96.89265536723164
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (cor-eng)
      config: cor-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 12.6
    - type: f1
      value: 9.981918087824628
    - type: precision
      value: 9.326319147606549
    - type: recall
      value: 12.6
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (cat-eng)
      config: cat-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.39999999999999
    - type: f1
      value: 96.65
    - type: precision
      value: 96.28333333333333
    - type: recall
      value: 97.39999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (eus-eng)
      config: eus-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.5
    - type: f1
      value: 95.38333333333333
    - type: precision
      value: 94.83333333333333
    - type: recall
      value: 96.5
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (yue-eng)
      config: yue-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 90.8
    - type: f1
      value: 88.43666666666665
    - type: precision
      value: 87.395
    - type: recall
      value: 90.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (swe-eng)
      config: swe-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.7
    - type: f1
      value: 97.03333333333333
    - type: precision
      value: 96.71666666666667
    - type: recall
      value: 97.7
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (dtp-eng)
      config: dtp-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 9.4
    - type: f1
      value: 7.946889105220061
    - type: precision
      value: 7.665059865752875
    - type: recall
      value: 9.4
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (kat-eng)
      config: kat-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.04021447721179
    - type: f1
      value: 93.68632707774799
    - type: precision
      value: 93.08534405719392
    - type: recall
      value: 95.04021447721179
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (jpn-eng)
      config: jpn-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.89999999999999
    - type: f1
      value: 94.66666666666667
    - type: precision
      value: 94.08333333333334
    - type: recall
      value: 95.89999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (csb-eng)
      config: csb-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 82.6086956521739
    - type: f1
      value: 77.98418972332016
    - type: precision
      value: 75.96837944664031
    - type: recall
      value: 82.6086956521739
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (xho-eng)
      config: xho-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.77464788732394
    - type: f1
      value: 94.8356807511737
    - type: precision
      value: 94.36619718309859
    - type: recall
      value: 95.77464788732394
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (orv-eng)
      config: orv-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 53.17365269461077
    - type: f1
      value: 47.07043056743655
    - type: precision
      value: 45.161363241830784
    - type: recall
      value: 53.17365269461077
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ind-eng)
      config: ind-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.5
    - type: f1
      value: 94.5
    - type: precision
      value: 94.03333333333333
    - type: recall
      value: 95.5
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (tuk-eng)
      config: tuk-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 93.59605911330048
    - type: f1
      value: 91.82266009852216
    - type: precision
      value: 91.09195402298852
    - type: recall
      value: 93.59605911330048
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (max-eng)
      config: max-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 76.40845070422534
    - type: f1
      value: 72.73082942097027
    - type: precision
      value: 71.46686939820742
    - type: recall
      value: 76.40845070422534
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (swh-eng)
      config: swh-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 93.58974358974359
    - type: f1
      value: 91.98290598290598
    - type: precision
      value: 91.3119658119658
    - type: recall
      value: 93.58974358974359
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (hin-eng)
      config: hin-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.8
    - type: f1
      value: 97.06666666666668
    - type: precision
      value: 96.7
    - type: recall
      value: 97.8
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (dsb-eng)
      config: dsb-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 68.89352818371609
    - type: f1
      value: 64.47860652453555
    - type: precision
      value: 62.878651918592574
    - type: recall
      value: 68.89352818371609
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ber-eng)
      config: ber-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 33.800000000000004
    - type: f1
      value: 29.290774344112368
    - type: precision
      value: 28.066016735704647
    - type: recall
      value: 33.800000000000004
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (tam-eng)
      config: tam-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 90.22801302931596
    - type: f1
      value: 88.07817589576547
    - type: precision
      value: 87.171552660152
    - type: recall
      value: 90.22801302931596
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (slk-eng)
      config: slk-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.2
    - type: f1
      value: 97.63333333333334
    - type: precision
      value: 97.36666666666667
    - type: recall
      value: 98.2
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (tgl-eng)
      config: tgl-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.7
    - type: f1
      value: 96.95
    - type: precision
      value: 96.58333333333331
    - type: recall
      value: 97.7
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ast-eng)
      config: ast-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 92.91338582677166
    - type: f1
      value: 90.81364829396327
    - type: precision
      value: 89.89501312335958
    - type: recall
      value: 92.91338582677166
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (mkd-eng)
      config: mkd-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.89999999999999
    - type: f1
      value: 95.98333333333332
    - type: precision
      value: 95.56666666666668
    - type: recall
      value: 96.89999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (khm-eng)
      config: khm-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 74.51523545706371
    - type: f1
      value: 70.20346919931407
    - type: precision
      value: 68.6389565788895
    - type: recall
      value: 74.51523545706371
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ces-eng)
      config: ces-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 97.6
    - type: f1
      value: 96.88333333333333
    - type: precision
      value: 96.53333333333333
    - type: recall
      value: 97.6
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (tzl-eng)
      config: tzl-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 46.15384615384615
    - type: f1
      value: 39.47885447885448
    - type: precision
      value: 37.301528599605525
    - type: recall
      value: 46.15384615384615
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (urd-eng)
      config: urd-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 94.69999999999999
    - type: f1
      value: 93.16666666666667
    - type: precision
      value: 92.41666666666667
    - type: recall
      value: 94.69999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (ara-eng)
      config: ara-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.19999999999999
    - type: f1
      value: 93.83333333333333
    - type: precision
      value: 93.16666666666667
    - type: recall
      value: 95.19999999999999
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (kor-eng)
      config: kor-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 92.0
    - type: f1
      value: 89.98666666666666
    - type: precision
      value: 89.09166666666667
    - type: recall
      value: 92.0
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (yid-eng)
      config: yid-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 95.51886792452831
    - type: f1
      value: 94.3003144654088
    - type: precision
      value: 93.75
    - type: recall
      value: 95.51886792452831
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (fin-eng)
      config: fin-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 98.2
    - type: f1
      value: 97.83333333333333
    - type: precision
      value: 97.65
    - type: recall
      value: 98.2
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (tha-eng)
      config: tha-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 96.8978102189781
    - type: f1
      value: 96.04622871046227
    - type: precision
      value: 95.62043795620438
    - type: recall
      value: 96.8978102189781
  - task:
      type: BitextMining
    dataset:
      type: mteb/tatoeba-bitext-mining
      name: MTEB Tatoeba (wuu-eng)
      config: wuu-eng
      split: test
      revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
    metrics:
    - type: accuracy
      value: 85.1
    - type: f1
      value: 81.78564213564214
    - type: precision
      value: 80.46416666666667
    - type: recall
      value: 85.1
  - task:
      type: Clustering
    dataset:
      type: slvnwhrl/tenkgnad-clustering-p2p
      name: MTEB TenKGnadClusteringP2P
      config: default
      split: test
      revision: 5c59e41555244b7e45c9a6be2d720ab4bafae558
    metrics:
    - type: v_measure
      value: 21.827519839402644
  - task:
      type: Clustering
    dataset:
      type: slvnwhrl/tenkgnad-clustering-s2s
      name: MTEB TenKGnadClusteringS2S
      config: default
      split: test
      revision: 6cddbe003f12b9b140aec477b583ac4191f01786
    metrics:
    - type: v_measure
      value: 27.160188241713684
  - task:
      type: Clustering
    dataset:
      type: C-MTEB/ThuNewsClusteringP2P
      name: MTEB ThuNewsClusteringP2P
      config: default
      split: test
      revision: 5798586b105c0434e4f0fe5e767abe619442cf93
    metrics:
    - type: v_measure
      value: 38.54459276932986
  - task:
      type: Clustering
    dataset:
      type: C-MTEB/ThuNewsClusteringS2S
      name: MTEB ThuNewsClusteringS2S
      config: default
      split: test
      revision: 8a8b2caeda43f39e13c4bc5bea0f8a667896e10d
    metrics:
    - type: v_measure
      value: 43.4460576234314
  - task:
      type: Retrieval
    dataset:
      type: webis-touche2020
      name: MTEB Touche2020
      config: default
      split: test
      revision: None
    metrics:
    - type: map_at_1
      value: 0.20500000000000002
    - type: map_at_10
      value: 0.391
    - type: map_at_100
      value: 0.612
    - type: map_at_1000
      value: 0.645
    - type: map_at_3
      value: 0.302
    - type: map_at_5
      value: 0.383
    - type: mrr_at_1
      value: 4.082
    - type: mrr_at_10
      value: 5.612
    - type: mrr_at_100
      value: 6.822
    - type: mrr_at_1000
      value: 6.929
    - type: mrr_at_3
      value: 4.082
    - type: mrr_at_5
      value: 5.408
    - type: ndcg_at_1
      value: 4.082
    - type: ndcg_at_10
      value: 1.6840000000000002
    - type: ndcg_at_100
      value: 2.876
    - type: ndcg_at_1000
      value: 4.114
    - type: ndcg_at_3
      value: 2.52
    - type: ndcg_at_5
      value: 2.3720000000000003
    - type: precision_at_1
      value: 4.082
    - type: precision_at_10
      value: 1.429
    - type: precision_at_100
      value: 0.755
    - type: precision_at_1000
      value: 0.18
    - type: precision_at_3
      value: 2.041
    - type: precision_at_5
      value: 2.4490000000000003
    - type: recall_at_1
      value: 0.20500000000000002
    - type: recall_at_10
      value: 0.761
    - type: recall_at_100
      value: 4.423
    - type: recall_at_1000
      value: 9.044
    - type: recall_at_3
      value: 0.302
    - type: recall_at_5
      value: 0.683
  - task:
      type: Classification
    dataset:
      type: mteb/toxic_conversations_50k
      name: MTEB ToxicConversationsClassification
      config: default
      split: test
      revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
    metrics:
    - type: accuracy
      value: 67.28359999999999
    - type: ap
      value: 12.424592214862038
    - type: f1
      value: 51.53630450055703
  - task:
      type: Classification
    dataset:
      type: mteb/tweet_sentiment_extraction
      name: MTEB TweetSentimentExtractionClassification
      config: default
      split: test
      revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
    metrics:
    - type: accuracy
      value: 56.23372948500284
    - type: f1
      value: 56.440924587214234
  - task:
      type: Clustering
    dataset:
      type: mteb/twentynewsgroups-clustering
      name: MTEB TwentyNewsgroupsClustering
      config: default
      split: test
      revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
    metrics:
    - type: v_measure
      value: 24.410059815620116
  - task:
      type: PairClassification
    dataset:
      type: mteb/twittersemeval2015-pairclassification
      name: MTEB TwitterSemEval2015
      config: default
      split: test
      revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
    metrics:
    - type: cos_sim_accuracy
      value: 80.3302139834297
    - type: cos_sim_ap
      value: 53.57723069745093
    - type: cos_sim_f1
      value: 51.58639580004565
    - type: cos_sim_precision
      value: 45.45454545454545
    - type: cos_sim_recall
      value: 59.63060686015831
    - type: dot_accuracy
      value: 80.3302139834297
    - type: dot_ap
      value: 53.57723006705641
    - type: dot_f1
      value: 51.58639580004565
    - type: dot_precision
      value: 45.45454545454545
    - type: dot_recall
      value: 59.63060686015831
    - type: euclidean_accuracy
      value: 80.3302139834297
    - type: euclidean_ap
      value: 53.57723050286929
    - type: euclidean_f1
      value: 51.58639580004565
    - type: euclidean_precision
      value: 45.45454545454545
    - type: euclidean_recall
      value: 59.63060686015831
    - type: manhattan_accuracy
      value: 80.31233235977827
    - type: manhattan_ap
      value: 53.44943961562638
    - type: manhattan_f1
      value: 51.24183006535947
    - type: manhattan_precision
      value: 43.63636363636363
    - type: manhattan_recall
      value: 62.05804749340369
    - type: max_accuracy
      value: 80.3302139834297
    - type: max_ap
      value: 53.57723069745093
    - type: max_f1
      value: 51.58639580004565
  - task:
      type: PairClassification
    dataset:
      type: mteb/twitterurlcorpus-pairclassification
      name: MTEB TwitterURLCorpus
      config: default
      split: test
      revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
    metrics:
    - type: cos_sim_accuracy
      value: 87.45876508712695
    - type: cos_sim_ap
      value: 83.5320716566614
    - type: cos_sim_f1
      value: 75.54560716284276
    - type: cos_sim_precision
      value: 73.27929362379678
    - type: cos_sim_recall
      value: 77.95657530027718
    - type: dot_accuracy
      value: 87.45876508712695
    - type: dot_ap
      value: 83.53209944887666
    - type: dot_f1
      value: 75.54560716284276
    - type: dot_precision
      value: 73.27929362379678
    - type: dot_recall
      value: 77.95657530027718
    - type: euclidean_accuracy
      value: 87.45876508712695
    - type: euclidean_ap
      value: 83.53205938307582
    - type: euclidean_f1
      value: 75.54560716284276
    - type: euclidean_precision
      value: 73.27929362379678
    - type: euclidean_recall
      value: 77.95657530027718
    - type: manhattan_accuracy
      value: 87.52280048123569
    - type: manhattan_ap
      value: 83.4884324728773
    - type: manhattan_f1
      value: 75.43366677906411
    - type: manhattan_precision
      value: 73.46566445303948
    - type: manhattan_recall
      value: 77.51000923929782
    - type: max_accuracy
      value: 87.52280048123569
    - type: max_ap
      value: 83.53209944887666
    - type: max_f1
      value: 75.54560716284276
  - task:
      type: Retrieval
    dataset:
      type: C-MTEB/VideoRetrieval
      name: MTEB VideoRetrieval
      config: default
      split: dev
      revision: 58c2597a5943a2ba48f4668c3b90d796283c5639
    metrics:
    - type: map_at_1
      value: 13.100000000000001
    - type: map_at_10
      value: 15.620000000000001
    - type: map_at_100
      value: 15.928
    - type: map_at_1000
      value: 15.976
    - type: map_at_3
      value: 14.817
    - type: map_at_5
      value: 15.322
    - type: mrr_at_1
      value: 13.0
    - type: mrr_at_10
      value: 15.57
    - type: mrr_at_100
      value: 15.878
    - type: mrr_at_1000
      value: 15.926000000000002
    - type: mrr_at_3
      value: 14.767
    - type: mrr_at_5
      value: 15.272
    - type: ndcg_at_1
      value: 13.100000000000001
    - type: ndcg_at_10
      value: 17.05
    - type: ndcg_at_100
      value: 18.801000000000002
    - type: ndcg_at_1000
      value: 20.436
    - type: ndcg_at_3
      value: 15.425
    - type: ndcg_at_5
      value: 16.333000000000002
    - type: precision_at_1
      value: 13.100000000000001
    - type: precision_at_10
      value: 2.16
    - type: precision_at_100
      value: 0.304
    - type: precision_at_1000
      value: 0.044000000000000004
    - type: precision_at_3
      value: 5.733
    - type: precision_at_5
      value: 3.88
    - type: recall_at_1
      value: 13.100000000000001
    - type: recall_at_10
      value: 21.6
    - type: recall_at_100
      value: 30.4
    - type: recall_at_1000
      value: 44.1
    - type: recall_at_3
      value: 17.2
    - type: recall_at_5
      value: 19.400000000000002
  - task:
      type: Classification
    dataset:
      type: C-MTEB/waimai-classification
      name: MTEB Waimai
      config: default
      split: test
      revision: 339287def212450dcaa9df8c22bf93e9980c7023
    metrics:
    - type: accuracy
      value: 76.12
    - type: ap
      value: 54.1619589378045
    - type: f1
      value: 74.32372858884229
  - task:
      type: Clustering
    dataset:
      type: jinaai/cities_wiki_clustering
      name: MTEB WikiCitiesClustering
      config: default
      split: test
      revision: ddc9ee9242fa65332597f70e967ecc38b9d734fa
    metrics:
    - type: v_measure
      value: 50.71744674029636
  - task:
      type: Retrieval
    dataset:
      type: jinaai/xmarket_de
      name: MTEB XMarketDE
      config: default
      split: test
      revision: 2336818db4c06570fcdf263e1bcb9993b786f67a
    metrics:
    - type: map_at_1
      value: 0.182
    - type: map_at_10
      value: 0.266
    - type: map_at_100
      value: 0.295
    - type: map_at_1000
      value: 0.313
    - type: map_at_3
      value: 0.232
    - type: map_at_5
      value: 0.23800000000000002
    - type: mrr_at_1
      value: 1.3379999999999999
    - type: mrr_at_10
      value: 1.918
    - type: mrr_at_100
      value: 2.051
    - type: mrr_at_1000
      value: 2.084
    - type: mrr_at_3
      value: 1.7049999999999998
    - type: mrr_at_5
      value: 1.791
    - type: ndcg_at_1
      value: 1.3379999999999999
    - type: ndcg_at_10
      value: 0.859
    - type: ndcg_at_100
      value: 0.8500000000000001
    - type: ndcg_at_1000
      value: 1.345
    - type: ndcg_at_3
      value: 1.032
    - type: ndcg_at_5
      value: 0.918
    - type: precision_at_1
      value: 1.3379999999999999
    - type: precision_at_10
      value: 0.528
    - type: precision_at_100
      value: 0.22699999999999998
    - type: precision_at_1000
      value: 0.132
    - type: precision_at_3
      value: 0.8829999999999999
    - type: precision_at_5
      value: 0.6890000000000001
    - type: recall_at_1
      value: 0.182
    - type: recall_at_10
      value: 0.51
    - type: recall_at_100
      value: 1.2229999999999999
    - type: recall_at_1000
      value: 4.183
    - type: recall_at_3
      value: 0.292
    - type: recall_at_5
      value: 0.315
---

# SONAR
[[Paper]](https://ai.meta.com/research/publications/sonar-sentence-level-multimodal-and-language-agnostic-representations/)

We introduce SONAR, a new multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders. It substantially outperforms existing sentence embeddings such as LASER3 and LabSE on the xsim and xsim++ multilingual similarity search tasks. 

Speech segments can be embedded in the same SONAR embedding space using language-specific speech encoders trained in a teacher-student setting on speech transcription data. We also provide a single text decoder, which allows us to perform text-to-text and speech-to-text machine translation, including for zero-shot language and modality combinations.

*SONAR* stands for **S**entence-level multim**O**dal and la**N**guage-**A**gnostic **R**epresentations

The full list of supported languages (along with download links) can be found here [below](#supported-languages-and-download-links).


## Installing
SONAR depends mainly on [Fairseq2](https://github.com/fairinternal/fairseq2) and can be installed using (tested with `python=3.8`)
```bash
pip install --upgrade pip
pip config set global.extra-index-url https://test.pypi.org/simple/
pip install -e .
```

## Usage
fairseq2 will automatically download models into your `$TORCH_HOME/hub` directory upon using the commands below.

### Compute text sentence embeddings with SONAR:
```python
from sonar.inference_pipelines.text import TextToEmbeddingModelPipeline
t2vec_model = TextToEmbeddingModelPipeline(encoder="text_sonar_basic_encoder",
                                           tokenizer="text_sonar_basic_encoder")
sentences = ['My name is SONAR.', 'I can embed the sentences into vectorial space.']
t2vec_model.predict(sentences, source_lang="eng_Latn").shape
# torch.Size([2, 1024])
```

### Translate text with SONAR
```python
from sonar.inference_pipelines.text import TextToTextModelPipeline
t2t_model = TextToTextModelPipeline(encoder="text_sonar_basic_encoder",
                                    decoder="text_sonar_basic_decoder",
                                    tokenizer="text_sonar_basic_encoder")  # tokenizer is attached to both encoder and decoder cards

sentences = ['My name is SONAR.', 'I can embed the sentences into vectorial space.']
t2t_model.predict(sentences, source_lang="eng_Latn", target_lang="fra_Latn")
# ['Mon nom est SONAR.', "Je peux intégrer les phrases dans l'espace vectoriel."]
```

### Compute speech sentence embeddings with SONAR
```python
from sonar.inference_pipelines.speech import SpeechToEmbeddingModelPipeline
s2vec_model = SpeechToEmbeddingModelPipeline(encoder="sonar_speech_encoder_eng")

s2vec_model.predict(["./tests/integration_tests/data/audio_files/audio_1.wav",
                     "./tests/integration_tests/data/audio_files/audio_2.wav"]).shape
# torch.Size([2, 1024])
import torchaudio
inp, sr = torchaudio.load("./tests/integration_tests/data/audio_files/audio_1.wav")
assert sr == 16000, "Sample rate should be 16kHz"

s2vec_model.predict([inp]).shape
# torch.Size([1, 1024])
```

### Speech-to-text translation with SONAR
```python
from sonar.inference_pipelines.speech import SpeechToTextModelPipeline

s2t_model = SpeechToTextModelPipeline(encoder="sonar_speech_encoder_eng",
                                      decoder="text_sonar_basic_decoder",
                                      tokenizer="text_sonar_basic_decoder")

import torchaudio
inp, sr = torchaudio.load("./tests/integration_tests/data/audio_files/audio_1.wav")
assert sr == 16000, "Sample rate should be 16kHz"

# passing loaded audio files
s2t_model.predict([inp], target_lang="eng_Latn")
# ['Television reports show white smoke coming from the plant.']

# passing multiple wav files 
s2t_model.predict(["./tests/integration_tests/data/audio_files/audio_1.wav",
                   "./tests/integration_tests/data/audio_files/audio_2.wav"], target_lang="eng_Latn")
# ['Television reports show white smoke coming from the plant.',
# 'These couples may choose to make an adoption plan for their baby.']
```


### Predicting [cross-lingual semantic similarity](https://github.com/facebookresearch/fairseq/tree/nllb/examples/nllb/human_XSTS_eval) with BLASER 2 models
```Python
import torch
from sonar.models.blaser.loader import load_blaser_model

blaser_ref = load_blaser_model("blaser_st2st_ref_v2_0").eval()
blaser_qe = load_blaser_model("blaser_st2st_qe_v2_0").eval()
# BLASER-2 is supposed to work with SONAR speech and text embeddings,
# but we didn't include their extraction in this snippet, to keep it simple.
emb = torch.ones([1, 1024])
print(blaser_ref(src=emb, ref=emb, mt=emb).item())  # 5.2552
print(blaser_qe(src=emb, mt=emb).item())  # 4.9819
```

See more complete demo notebooks :

* [sonar text2text similarity and translation](examples/sonar_text_demo.ipynb)
* [sonar speech2text and other data pipeline examples](examples/inference_pipelines.ipynb)


## Model details

- **Developed by:** Paul-Ambroise Duquenne et al.
- **License:** CC-BY-NC 4.0 license
- **Cite as:**

```
  @article{Duquenne:2023:sonar_arxiv,
    author = {Paul-Ambroise Duquenne and Holger Schwenk and Benoit Sagot},
    title = {{SONAR:} Sentence-Level Multimodal and Language-Agnostic Representations},
    publisher = {arXiv},
    year = {2023},
    url = {https://arxiv.org/abs/unk},
  }
```