jarredparrett/all-MiniLM-L6-v2_tuned_on_deepparse_address_mutations_comb_3

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the deepparse_address_mutations_comb_3 dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity
• Training Dataset:
  • deepparse_address_mutations_comb_3
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jarredparrett/all-MiniLM-L6-v2_tuned_on_deepparse_address_mutations_comb_3")
# Run inference
sentences = [
    '8234 harvest bend lane laurel md 20707',
    '8234 harvest bend lane laurel md',
    '8702 wahl crse basement santee ca 92071',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Binary Classification

• Dataset: jarredparrett/deepparse_address_mutations_comb_3
• Evaluated with BinaryClassificationEvaluator

Metric	Value
cosine_accuracy	0.9771
cosine_accuracy_threshold	0.7712
cosine_f1	0.9784
cosine_f1_threshold	0.7712
cosine_precision	0.9601
cosine_recall	0.9974
cosine_ap	0.9865
dot_accuracy	0.9771
dot_accuracy_threshold	0.7712
dot_f1	0.9784
dot_f1_threshold	0.7712
dot_precision	0.9601
dot_recall	0.9974
dot_ap	0.9865
manhattan_accuracy	0.977
manhattan_accuracy_threshold	10.6015
manhattan_f1	0.9784
manhattan_f1_threshold	10.6118
manhattan_precision	0.96
manhattan_recall	0.9974
manhattan_ap	0.9865
euclidean_accuracy	0.9771
euclidean_accuracy_threshold	0.6764
euclidean_f1	0.9784
euclidean_f1_threshold	0.6764
euclidean_precision	0.9601
euclidean_recall	0.9974
euclidean_ap	0.9866
max_accuracy	0.9771
max_accuracy_threshold	10.6015
max_f1	0.9784
max_f1_threshold	10.6118
max_precision	0.9601
max_recall	0.9974
max_ap	0.9866

Binary Classification

• Dataset: jarredparrett/deepparse_address_mutations_comb_3
• Evaluated with BinaryClassificationEvaluator

Metric	Value
cosine_accuracy	0.9771
cosine_accuracy_threshold	0.7711
cosine_f1	0.9784
cosine_f1_threshold	0.7711
cosine_precision	0.96
cosine_recall	0.9974
cosine_ap	0.9865
dot_accuracy	0.9771
dot_accuracy_threshold	0.7711
dot_f1	0.9784
dot_f1_threshold	0.7711
dot_precision	0.96
dot_recall	0.9974
dot_ap	0.9866
manhattan_accuracy	0.977
manhattan_accuracy_threshold	10.5101
manhattan_f1	0.9784
manhattan_f1_threshold	10.6372
manhattan_precision	0.9599
manhattan_recall	0.9975
manhattan_ap	0.9866
euclidean_accuracy	0.9771
euclidean_accuracy_threshold	0.6766
euclidean_f1	0.9784
euclidean_f1_threshold	0.6766
euclidean_precision	0.96
euclidean_recall	0.9974
euclidean_ap	0.9866
max_accuracy	0.9771
max_accuracy_threshold	10.5101
max_f1	0.9784
max_f1_threshold	10.6372
max_precision	0.96
max_recall	0.9975
max_ap	0.9866

Training Details

Training Dataset

deepparse_address_mutations_comb_3

• Dataset: deepparse_address_mutations_comb_3 at 7162fdc
• Size: 4,517,388 training samples
• Columns: label, sentence1, and sentence2
• Approximate statistics based on the first 1000 samples:

  	label	sentence1	sentence2
  type	torch.Tensor	string	string
  details		min: 8 tokens mean: 13.21 tokens max: 22 tokens	min: 6 tokens mean: 13.54 tokens max: 22 tokens

• Samples:

  label	sentence1	sentence2
  tensor(1, device='cuda:0')	12737 chesdin landng dr chesterfield va 23838	12737 chesdin landng dr chesterfield va
  tensor(1, device='cuda:0')	6080 norh oak trafficway gladstone mo 64118	6080 norh oak trafficway gladstone 64118-4896
  tensor(0, device='cuda:0')	242 pierce view cir wentzville mo 63385	242 pierce view cir wentzville LA 63385

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Evaluation Dataset

deepparse_address_mutations_comb_3

• Dataset: deepparse_address_mutations_comb_3 at 7162fdc
• Size: 968,012 evaluation samples
• Columns: label, sentence1, and sentence2
• Approximate statistics based on the first 1000 samples:

  	label	sentence1	sentence2
  type	torch.Tensor	string	string
  details		min: 8 tokens mean: 13.24 tokens max: 22 tokens	min: 7 tokens mean: 13.45 tokens max: 27 tokens

• Samples:

  label	sentence1	sentence2
  tensor(1, device='cuda:0')	1 vincent avenue essex maryland 21221	1 vincent avenue essedx MD 21221
  tensor(1, device='cuda:0')	139 berg avenue hamilton tshp n.j. 08610	139 bcrg avenue hamilton tshp n.j. 08610
  tensor(1, device='cuda:0')	714 havard rd houston texas 77336	714 havaplns plns houston texas 77336-3120

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 1024
• per_device_eval_batch_size: 1024
• learning_rate: 2e-05
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 1024
• per_device_eval_batch_size: 1024
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	loss	jarredparrett/deepparse_address_mutations_comb_3_max_ap
0.1133	500	0.0191	0.0131	0.8459
0.2267	1000	0.0112	0.0091	0.8887
0.3400	1500	0.0086	0.0067	0.9346
0.4533	2000	0.0064	0.0044	0.9604
0.5666	2500	0.0049	0.0037	0.9722
0.6800	3000	0.0042	0.0033	0.9761
0.7933	3500	0.0039	0.0032	0.9808
0.9066	4000	0.0037	0.0029	0.9825
1.0197	4500	0.0035	0.0028	0.9826
1.1330	5000	0.0033	0.0028	0.9836
1.2464	5500	0.0032	0.0027	0.9845
1.3597	6000	0.0031	0.0026	0.9853
1.4730	6500	0.003	0.0025	0.9857
1.5864	7000	0.003	0.0025	0.9859
1.6997	7500	0.0029	0.0025	0.9862
1.8130	8000	0.0028	0.0024	0.9864
1.9263	8500	0.0028	0.0024	0.9861
2.0394	9000	0.0028	0.0024	0.9864
2.1528	9500	0.0027	0.0024	0.9864
2.2661	10000	0.0027	0.0024	0.9865
2.3794	10500	0.0027	0.0023	0.9866
2.4927	11000	0.0026	0.0023	0.9866
2.6061	11500	0.0026	0.0023	0.9865
2.7194	12000	0.0026	0.0023	0.9865
2.8327	12500	0.0026	0.0023	0.9865
2.9461	13000	0.0026	0.0023	0.9866
2.9995	13236	-	-	0.9866

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.1.1
• Transformers: 4.45.2
• PyTorch: 2.5.1+cu121
• Accelerate: 1.1.1
• Datasets: 3.2.0
• Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveLoss

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}