Hugging Face's logo

manuel-couto-pintos
/
roberta_erisk_sts

Sentence Similarity
sentence-transformers
Safetensors
roberta
feature-extraction
Generated from Trainer
dataset_size:50881
loss:TripletLoss
text-embeddings-inference
Inference Endpoints
Model card Files Community
roberta_erisk_sts / README.md
manuel-couto-pintos's picture
manuel-couto-pintos
Add new SentenceTransformer model.
58a81db verified
preview code
|
raw
history blame contribute delete
16.3 kB
metadata
base_model: manuel-couto-pintos/roberta_erisk
datasets: []
language: []
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:50881
  - loss:TripletLoss
widget:
  - source_sentence: >-
      I smoked weed for the first time ever a couple days ago, how long until
      it's out of my system?
    sentences:
      - >-
        If I haven't smoked weed in a long time and smoked 1 day, how long will
        it be in my urine?
      - Where can we find best delay pedal?
      - How long does it take for an avid weed smoker to pass a urine drug test?
  - source_sentence: What are the visiting places in coorg?
    sentences:
      - How can I find a co-working space in Gurgaon?
      - What are the places to visit in coorg?
      - What are your favourite celebrity cookbooks?
  - source_sentence: What is the best used car to get under 5k?
    sentences:
      - What's the best used car for under 5k?
      - What do you think about RBI's new move of banning 500 and 1000 notes?
      - Which is the best car to buy under 6 lakhs?
  - source_sentence: Which exercises can I do at home to reduce belly fat?
    sentences:
      - What exercise we can do to reduce belly fat at home?
      - What is a first time home buyer?
      - >-
        My upper body is in shape but my thighs are very fatty and big ...so how
        can I reduce my thighs .I am doing running of 3km daily only?
  - source_sentence: Which is the best affiliate program?
    sentences:
      - How can I learn to make good coffee at home?
      - What are the best affiliate networks in the UK?
      - What are the best affiliate programs?

SentenceTransformer based on manuel-couto-pintos/roberta_erisk

This is a sentence-transformers model finetuned from manuel-couto-pintos/roberta_erisk. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: manuel-couto-pintos/roberta_erisk
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 768 tokens
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("manuel-couto-pintos/roberta_erisk_sts")
# Run inference
sentences = [
    'Which is the best affiliate program?',
    'What are the best affiliate programs?',
    'What are the best affiliate networks in the UK?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

  • Size: 50,881 training samples
  • Columns: sentence_0, sentence_1, and sentence_2
  • Approximate statistics based on the first 1000 samples:
    sentence_0 sentence_1 sentence_2
    type string string string
    details
    • min: 6 tokens
    • mean: 13.77 tokens
    • max: 42 tokens
    • min: 6 tokens
    • mean: 13.82 tokens
    • max: 57 tokens
    • min: 6 tokens
    • mean: 14.96 tokens
    • max: 59 tokens
  • Samples:
    sentence_0 sentence_1 sentence_2
    What is a good definition of Quora? What is the best definition of Quora? What is Quora address?
    How can I make myself appear offline on facebook? How do you make sure to appear as offline on Facebook? How can I get Facebook to remember to keep chat offline?
    How do I gain some healthy weight? What is the best way for underweight to gain weight? My boyfriend doesn't eat a lot. What are some ways to help him gain weight fast? He's 5'7 120lbs
  • Loss: TripletLoss with these parameters:
    {
    "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
    "triplet_margin": 5
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 10
  • per_device_eval_batch_size: 10
  • num_train_epochs: 10
  • multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: no
  • prediction_loss_only: True
  • per_device_train_batch_size: 10
  • per_device_eval_batch_size: 10
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 5e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1
  • num_train_epochs: 10
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.0
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • eval_use_gather_object: False
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: round_robin

Training Logs

Click to expand
Epoch Step Training Loss
0.0983 500 4.3807
0.1965 1000 2.5872
0.2948 1500 1.7484
0.3930 2000 1.2649
0.4913 2500 1.0219
0.5895 3000 0.8703
0.6878 3500 0.771
0.7860 4000 0.655
0.8843 4500 0.6547
0.9825 5000 0.5772
1.0808 5500 0.5628
1.1790 6000 0.5163
1.2773 6500 0.4871
1.3755 7000 0.4842
1.4738 7500 0.4316
1.5720 8000 0.4199
1.6703 8500 0.3554
1.7685 9000 0.3467
1.8668 9500 0.3591
1.9650 10000 0.3356
2.0633 10500 0.3281
2.1615 11000 0.3149
2.2598 11500 0.2767
2.3580 12000 0.2849
2.4563 12500 0.244
2.5545 13000 0.2416
2.6528 13500 0.2008
2.7510 14000 0.1718
2.8493 14500 0.188
2.9475 15000 0.1656
3.0458 15500 0.1522
3.1440 16000 0.144
3.2423 16500 0.1329
3.3405 17000 0.1431
3.4388 17500 0.128
3.5370 18000 0.1251
3.6353 18500 0.0921
3.7335 19000 0.0882
3.8318 19500 0.1087
3.9300 20000 0.0819
4.0283 20500 0.0916
4.1265 21000 0.0837
4.2248 21500 0.0855
4.3230 22000 0.0727
4.4213 22500 0.0772
4.5196 23000 0.0676
4.6178 23500 0.0597
4.7161 24000 0.0555
4.8143 24500 0.0613
4.9126 25000 0.0589
5.0108 25500 0.0503
5.1091 26000 0.0546
5.2073 26500 0.0446
5.3056 27000 0.0591
5.4038 27500 0.0431
5.5021 28000 0.0402
5.6003 28500 0.0354
5.6986 29000 0.0405
5.7968 29500 0.0308
5.8951 30000 0.0363
5.9933 30500 0.0365
6.0916 31000 0.0333
6.1898 31500 0.0238
6.2881 32000 0.0372
6.3863 32500 0.0331
6.4846 33000 0.0253
6.5828 33500 0.0315
6.6811 34000 0.0193
6.7793 34500 0.0239
6.8776 35000 0.0201
6.9758 35500 0.0213
7.0741 36000 0.0187
7.1723 36500 0.0125
7.2706 37000 0.0151
7.3688 37500 0.0208
7.4671 38000 0.0101
7.5653 38500 0.0191
7.6636 39000 0.0125
7.7618 39500 0.0136
7.8601 40000 0.0135
7.9583 40500 0.0118
8.0566 41000 0.012
8.1548 41500 0.0079
8.2531 42000 0.0105
8.3513 42500 0.0094
8.4496 43000 0.0079
8.5478 43500 0.0118
8.6461 44000 0.0105
8.7444 44500 0.0058
8.8426 45000 0.013
8.9409 45500 0.0065
9.0391 46000 0.0089
9.1374 46500 0.0031
9.2356 47000 0.008
9.3339 47500 0.0065
9.4321 48000 0.0052
9.5304 48500 0.0066
9.6286 49000 0.0039
9.7269 49500 0.004
9.8251 50000 0.0051
9.9234 50500 0.003

Framework Versions

  • Python: 3.10.14
  • Sentence Transformers: 3.0.1
  • Transformers: 4.44.2
  • PyTorch: 2.0.1+cu117
  • Accelerate: 0.32.0
  • Datasets: 2.20.0
  • Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss 

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification}, 
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}