luanafelbarros commited on
Commit
a70bb02
·
verified ·
1 Parent(s): 016a113

Add new SentenceTransformer model

Browse files
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,482 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - generated_from_trainer
7
+ - dataset_size:2560698
8
+ - loss:ModifiedMatryoshkaLoss
9
+ base_model: google-bert/bert-base-multilingual-cased
10
+ widget:
11
+ - source_sentence: We got off the exit, we found a Shoney's restaurant.
12
+ sentences:
13
+ - Nos alejamos de la salida, comenzamos a buscar un -- encontramos un restaurante
14
+ Shoney's.
15
+ - Reduzcan sus emisiones de dióxido de carbono con todo el rango de opciones que
16
+ tienen y luego compren o adquieran compensaciones para el resto que no han reducido
17
+ completamente.
18
+ - En el momento que nos invitaron a ir allí teníamos sede en San Francisco. Así
19
+ que fuimos de un lado a otro durante el resto de 2009, pasando la mitad del tiempo
20
+ en el condado de Bertie.
21
+ - source_sentence: And in the audio world that's when the microphone gets too close
22
+ to its sound source, and then it gets in this self-destructive loop that creates
23
+ a very unpleasant sound.
24
+ sentences:
25
+ - Y, en el mundo del audio, es cuando el micrófono se acerca demasiado a su fuente
26
+ de sonido, y entra en este bucle autodestructivo que crea un sonido muy desagradable.
27
+ - Tiene que ayudarles a alcanzar un compromiso equitativo, y a asegurar que una
28
+ amplia coalición de partidarios locales regionales e internacionales les ayuden
29
+ a implementar el acuerdo.
30
+ - Y es un renegado y visionario absoluto, y esa es la razón por la que ahora vivo
31
+ y trabajo allí.
32
+ - source_sentence: Figure out some of the other options that are much better.
33
+ sentences:
34
+ - Así que no sólo estamos reclutando a las multinacionales, les estamos dando las
35
+ herramientas para entregar este bien público, el respeto por los Derechos Humanos,
36
+ y lo estamos verificando.
37
+ - Piensen en otras de las opciones que son mucho mejores.
38
+ - Termina la propiedad comunal de las tierras de reserva.
39
+ - source_sentence: He is 16 years old, loves hunting and fishing and being outside
40
+ and doing anything with his hands, and so for him, Studio H means that he can
41
+ stay interested in his education through that hands-on engagement.
42
+ sentences:
43
+ - Tiene 16 años, le encanta cazar, pescar y estar al aire libre y hacer tareas manuales.
44
+ Para él Studio H representa el nexo educativo mediante esa motivación práctica.
45
+ - Carbón capturado y secuestrado -- eso es lo que CCS significa -- es probable que
46
+ se convierta en la aplicación determinante que nos posibilitará continuar utilizando
47
+ combustibles fósiles en un modo que sea seguro.
48
+ - El condado de Bertie no es la excepción.
49
+ - source_sentence: There are thousands of these blue dots all over the county.
50
+ sentences:
51
+ - Me gusta crisis climática en vez de colapso climático, pero de nuevo, aquellos
52
+ de ustedes que son buenos en diseño de marcas, necesito su ayuda en esto.
53
+ - Si miran con cuidado, se ve que su cráneo ha sido sustituido por una cúpula transparente
54
+ de plexiglás así que el funcionamiento de su cerebro se puede observar y controlar
55
+ con luz.
56
+ - Hay miles de estos puntos azules en todo el condado.
57
+ pipeline_tag: sentence-similarity
58
+ library_name: sentence-transformers
59
+ metrics:
60
+ - negative_mse
61
+ model-index:
62
+ - name: SentenceTransformer based on google-bert/bert-base-multilingual-cased
63
+ results:
64
+ - task:
65
+ type: knowledge-distillation
66
+ name: Knowledge Distillation
67
+ dataset:
68
+ name: MSE val en es
69
+ type: MSE-val-en-es
70
+ metrics:
71
+ - type: negative_mse
72
+ value: -31.070706248283386
73
+ name: Negative Mse
74
+ - task:
75
+ type: knowledge-distillation
76
+ name: Knowledge Distillation
77
+ dataset:
78
+ name: MSE val en pt
79
+ type: MSE-val-en-pt
80
+ metrics:
81
+ - type: negative_mse
82
+ value: -31.284737586975098
83
+ name: Negative Mse
84
+ - task:
85
+ type: knowledge-distillation
86
+ name: Knowledge Distillation
87
+ dataset:
88
+ name: MSE val en pt br
89
+ type: MSE-val-en-pt-br
90
+ metrics:
91
+ - type: negative_mse
92
+ value: -29.748335480690002
93
+ name: Negative Mse
94
+ ---
95
+
96
+ # SentenceTransformer based on google-bert/bert-base-multilingual-cased
97
+
98
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [google-bert/bert-base-multilingual-cased](https://huggingface.co/google-bert/bert-base-multilingual-cased). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
99
+
100
+ ## Model Details
101
+
102
+ ### Model Description
103
+ - **Model Type:** Sentence Transformer
104
+ - **Base model:** [google-bert/bert-base-multilingual-cased](https://huggingface.co/google-bert/bert-base-multilingual-cased) <!-- at revision 3f076fdb1ab68d5b2880cb87a0886f315b8146f8 -->
105
+ - **Maximum Sequence Length:** 128 tokens
106
+ - **Output Dimensionality:** 768 dimensions
107
+ - **Similarity Function:** Cosine Similarity
108
+ <!-- - **Training Dataset:** Unknown -->
109
+ <!-- - **Language:** Unknown -->
110
+ <!-- - **License:** Unknown -->
111
+
112
+ ### Model Sources
113
+
114
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
115
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
116
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
117
+
118
+ ### Full Model Architecture
119
+
120
+ ```
121
+ SentenceTransformer(
122
+ (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
123
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
124
+ )
125
+ ```
126
+
127
+ ## Usage
128
+
129
+ ### Direct Usage (Sentence Transformers)
130
+
131
+ First install the Sentence Transformers library:
132
+
133
+ ```bash
134
+ pip install -U sentence-transformers
135
+ ```
136
+
137
+ Then you can load this model and run inference.
138
+ ```python
139
+ from sentence_transformers import SentenceTransformer
140
+
141
+ # Download from the 🤗 Hub
142
+ model = SentenceTransformer("luanafelbarros/bert-en-es-pt-matryoshka_v1")
143
+ # Run inference
144
+ sentences = [
145
+ 'There are thousands of these blue dots all over the county.',
146
+ 'Hay miles de estos puntos azules en todo el condado.',
147
+ 'Me gusta crisis climática en vez de colapso climático, pero de nuevo, aquellos de ustedes que son buenos en diseño de marcas, necesito su ayuda en esto.',
148
+ ]
149
+ embeddings = model.encode(sentences)
150
+ print(embeddings.shape)
151
+ # [3, 768]
152
+
153
+ # Get the similarity scores for the embeddings
154
+ similarities = model.similarity(embeddings, embeddings)
155
+ print(similarities.shape)
156
+ # [3, 3]
157
+ ```
158
+
159
+ <!--
160
+ ### Direct Usage (Transformers)
161
+
162
+ <details><summary>Click to see the direct usage in Transformers</summary>
163
+
164
+ </details>
165
+ -->
166
+
167
+ <!--
168
+ ### Downstream Usage (Sentence Transformers)
169
+
170
+ You can finetune this model on your own dataset.
171
+
172
+ <details><summary>Click to expand</summary>
173
+
174
+ </details>
175
+ -->
176
+
177
+ <!--
178
+ ### Out-of-Scope Use
179
+
180
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
181
+ -->
182
+
183
+ ## Evaluation
184
+
185
+ ### Metrics
186
+
187
+ #### Knowledge Distillation
188
+
189
+ * Datasets: `MSE-val-en-es`, `MSE-val-en-pt` and `MSE-val-en-pt-br`
190
+ * Evaluated with [<code>MSEEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.MSEEvaluator)
191
+
192
+ | Metric | MSE-val-en-es | MSE-val-en-pt | MSE-val-en-pt-br |
193
+ |:-----------------|:--------------|:--------------|:-----------------|
194
+ | **negative_mse** | **-31.0707** | **-31.2847** | **-29.7483** |
195
+
196
+ <!--
197
+ ## Bias, Risks and Limitations
198
+
199
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
200
+ -->
201
+
202
+ <!--
203
+ ### Recommendations
204
+
205
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
206
+ -->
207
+
208
+ ## Training Details
209
+
210
+ ### Training Dataset
211
+
212
+ #### Unnamed Dataset
213
+
214
+
215
+ * Size: 2,560,698 training samples
216
+ * Columns: <code>english</code>, <code>non_english</code>, and <code>label</code>
217
+ * Approximate statistics based on the first 1000 samples:
218
+ | | english | non_english | label |
219
+ |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-------------------------------------|
220
+ | type | string | string | list |
221
+ | details | <ul><li>min: 4 tokens</li><li>mean: 25.46 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 26.67 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>size: 768 elements</li></ul> |
222
+ * Samples:
223
+ | english | non_english | label |
224
+ |:-------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------|
225
+ | <code>And then there are certain conceptual things that can also benefit from hand calculating, but I think they're relatively small in number.</code> | <code>Y luego hay ciertas aspectos conceptuales que pueden beneficiarse del cálculo a mano pero creo que son relativamente pocos.</code> | <code>[-0.015244179405272007, 0.04601434990763664, -0.052873335778713226, 0.03535117208957672, -0.039562877267599106, ...]</code> |
226
+ | <code>One thing I often ask about is ancient Greek and how this relates.</code> | <code>Algo que pregunto a menudo es sobre el griego antiguo y cómo se relaciona.</code> | <code>[0.0012022971641272306, -0.009590390138328075, -0.032977133989334106, 0.017047710716724396, -0.0028919472824782133, ...]</code> |
227
+ | <code>See, the thing we're doing right now is we're forcing people to learn mathematics.</code> | <code>Vean, lo que estamos haciendo ahora es forzar a la gente a aprender matemáticas.</code> | <code>[-0.01942082867026329, 0.1043599545955658, 0.009455358609557152, -0.02814248949289322, -0.017036128789186478, ...]</code> |
228
+ * Loss: <code>__main__.ModifiedMatryoshkaLoss</code> with these parameters:
229
+ ```json
230
+ {
231
+ "loss": "MSELoss",
232
+ "matryoshka_dims": [
233
+ 768,
234
+ 512,
235
+ 256,
236
+ 128,
237
+ 64
238
+ ],
239
+ "matryoshka_weights": [
240
+ 1,
241
+ 1,
242
+ 1,
243
+ 1,
244
+ 1
245
+ ],
246
+ "n_dims_per_step": -1
247
+ }
248
+ ```
249
+
250
+ ### Evaluation Dataset
251
+
252
+ #### Unnamed Dataset
253
+
254
+
255
+ * Size: 6,974 evaluation samples
256
+ * Columns: <code>english</code>, <code>non_english</code>, and <code>label</code>
257
+ * Approximate statistics based on the first 1000 samples:
258
+ | | english | non_english | label |
259
+ |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-------------------------------------|
260
+ | type | string | string | list |
261
+ | details | <ul><li>min: 4 tokens</li><li>mean: 25.68 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 27.31 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>size: 768 elements</li></ul> |
262
+ * Samples:
263
+ | english | non_english | label |
264
+ |:-----------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------|
265
+ | <code>Thank you so much, Chris.</code> | <code>Muchas gracias Chris.</code> | <code>[-0.0616779625415802, -0.04450426995754242, -0.03250579163432121, -0.06641441583633423, 0.003981655463576317, ...]</code> |
266
+ | <code>And it's truly a great honor to have the opportunity to come to this stage twice; I'm extremely grateful.</code> | <code>Y es en verdad un gran honor tener la oportunidad de venir a este escenario por segunda vez. Estoy extremadamente agradecido.</code> | <code>[0.011398598551750183, -0.02500401996076107, -0.009884790517389774, 0.009336900897324085, 0.003082842566072941, ...]</code> |
267
+ | <code>I have been blown away by this conference, and I want to thank all of you for the many nice comments about what I had to say the other night.</code> | <code>He quedado conmovido por esta conferencia, y deseo agradecer a todos ustedes sus amables comentarios acerca de lo que tenía que decir la otra noche.</code> | <code>[-0.03842132166028023, 0.03635749593377113, -0.02491452544927597, -0.0032229204662144184, 0.0003549510147422552, ...]</code> |
268
+ * Loss: <code>__main__.ModifiedMatryoshkaLoss</code> with these parameters:
269
+ ```json
270
+ {
271
+ "loss": "MSELoss",
272
+ "matryoshka_dims": [
273
+ 768,
274
+ 512,
275
+ 256,
276
+ 128,
277
+ 64
278
+ ],
279
+ "matryoshka_weights": [
280
+ 1,
281
+ 1,
282
+ 1,
283
+ 1,
284
+ 1
285
+ ],
286
+ "n_dims_per_step": -1
287
+ }
288
+ ```
289
+
290
+ ### Training Hyperparameters
291
+ #### Non-Default Hyperparameters
292
+
293
+ - `eval_strategy`: steps
294
+ - `per_device_train_batch_size`: 200
295
+ - `per_device_eval_batch_size`: 200
296
+ - `learning_rate`: 2e-05
297
+ - `num_train_epochs`: 1
298
+ - `warmup_ratio`: 0.1
299
+ - `fp16`: True
300
+ - `label_names`: ['label']
301
+
302
+ #### All Hyperparameters
303
+ <details><summary>Click to expand</summary>
304
+
305
+ - `overwrite_output_dir`: False
306
+ - `do_predict`: False
307
+ - `eval_strategy`: steps
308
+ - `prediction_loss_only`: True
309
+ - `per_device_train_batch_size`: 200
310
+ - `per_device_eval_batch_size`: 200
311
+ - `per_gpu_train_batch_size`: None
312
+ - `per_gpu_eval_batch_size`: None
313
+ - `gradient_accumulation_steps`: 1
314
+ - `eval_accumulation_steps`: None
315
+ - `torch_empty_cache_steps`: None
316
+ - `learning_rate`: 2e-05
317
+ - `weight_decay`: 0.0
318
+ - `adam_beta1`: 0.9
319
+ - `adam_beta2`: 0.999
320
+ - `adam_epsilon`: 1e-08
321
+ - `max_grad_norm`: 1.0
322
+ - `num_train_epochs`: 1
323
+ - `max_steps`: -1
324
+ - `lr_scheduler_type`: linear
325
+ - `lr_scheduler_kwargs`: {}
326
+ - `warmup_ratio`: 0.1
327
+ - `warmup_steps`: 0
328
+ - `log_level`: passive
329
+ - `log_level_replica`: warning
330
+ - `log_on_each_node`: True
331
+ - `logging_nan_inf_filter`: True
332
+ - `save_safetensors`: True
333
+ - `save_on_each_node`: False
334
+ - `save_only_model`: False
335
+ - `restore_callback_states_from_checkpoint`: False
336
+ - `no_cuda`: False
337
+ - `use_cpu`: False
338
+ - `use_mps_device`: False
339
+ - `seed`: 42
340
+ - `data_seed`: None
341
+ - `jit_mode_eval`: False
342
+ - `use_ipex`: False
343
+ - `bf16`: False
344
+ - `fp16`: True
345
+ - `fp16_opt_level`: O1
346
+ - `half_precision_backend`: auto
347
+ - `bf16_full_eval`: False
348
+ - `fp16_full_eval`: False
349
+ - `tf32`: None
350
+ - `local_rank`: 0
351
+ - `ddp_backend`: None
352
+ - `tpu_num_cores`: None
353
+ - `tpu_metrics_debug`: False
354
+ - `debug`: []
355
+ - `dataloader_drop_last`: False
356
+ - `dataloader_num_workers`: 0
357
+ - `dataloader_prefetch_factor`: None
358
+ - `past_index`: -1
359
+ - `disable_tqdm`: False
360
+ - `remove_unused_columns`: True
361
+ - `label_names`: ['label']
362
+ - `load_best_model_at_end`: False
363
+ - `ignore_data_skip`: False
364
+ - `fsdp`: []
365
+ - `fsdp_min_num_params`: 0
366
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
367
+ - `fsdp_transformer_layer_cls_to_wrap`: None
368
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
369
+ - `deepspeed`: None
370
+ - `label_smoothing_factor`: 0.0
371
+ - `optim`: adamw_torch
372
+ - `optim_args`: None
373
+ - `adafactor`: False
374
+ - `group_by_length`: False
375
+ - `length_column_name`: length
376
+ - `ddp_find_unused_parameters`: None
377
+ - `ddp_bucket_cap_mb`: None
378
+ - `ddp_broadcast_buffers`: False
379
+ - `dataloader_pin_memory`: True
380
+ - `dataloader_persistent_workers`: False
381
+ - `skip_memory_metrics`: True
382
+ - `use_legacy_prediction_loop`: False
383
+ - `push_to_hub`: False
384
+ - `resume_from_checkpoint`: None
385
+ - `hub_model_id`: None
386
+ - `hub_strategy`: every_save
387
+ - `hub_private_repo`: False
388
+ - `hub_always_push`: False
389
+ - `gradient_checkpointing`: False
390
+ - `gradient_checkpointing_kwargs`: None
391
+ - `include_inputs_for_metrics`: False
392
+ - `include_for_metrics`: []
393
+ - `eval_do_concat_batches`: True
394
+ - `fp16_backend`: auto
395
+ - `push_to_hub_model_id`: None
396
+ - `push_to_hub_organization`: None
397
+ - `mp_parameters`:
398
+ - `auto_find_batch_size`: False
399
+ - `full_determinism`: False
400
+ - `torchdynamo`: None
401
+ - `ray_scope`: last
402
+ - `ddp_timeout`: 1800
403
+ - `torch_compile`: False
404
+ - `torch_compile_backend`: None
405
+ - `torch_compile_mode`: None
406
+ - `dispatch_batches`: None
407
+ - `split_batches`: None
408
+ - `include_tokens_per_second`: False
409
+ - `include_num_input_tokens_seen`: False
410
+ - `neftune_noise_alpha`: None
411
+ - `optim_target_modules`: None
412
+ - `batch_eval_metrics`: False
413
+ - `eval_on_start`: False
414
+ - `use_liger_kernel`: False
415
+ - `eval_use_gather_object`: False
416
+ - `average_tokens_across_devices`: False
417
+ - `prompts`: None
418
+ - `batch_sampler`: batch_sampler
419
+ - `multi_dataset_batch_sampler`: proportional
420
+
421
+ </details>
422
+
423
+ ### Training Logs
424
+ | Epoch | Step | Training Loss | Validation Loss | MSE-val-en-es_negative_mse | MSE-val-en-pt_negative_mse | MSE-val-en-pt-br_negative_mse |
425
+ |:------:|:-----:|:-------------:|:---------------:|:--------------------------:|:--------------------------:|:-----------------------------:|
426
+ | 0.0781 | 1000 | 0.0252 | 0.0231 | -24.4152 | -24.3443 | -25.3002 |
427
+ | 0.1562 | 2000 | 0.0222 | 0.0212 | -25.3038 | -25.3995 | -24.8563 |
428
+ | 0.2343 | 3000 | 0.021 | 0.0204 | -27.0894 | -27.2195 | -26.2906 |
429
+ | 0.3124 | 4000 | 0.0204 | 0.0198 | -28.7895 | -28.9815 | -28.0121 |
430
+ | 0.3905 | 5000 | 0.02 | 0.0194 | -29.1917 | -29.3694 | -28.0828 |
431
+ | 0.4686 | 6000 | 0.0196 | 0.0191 | -30.0902 | -30.2569 | -28.9723 |
432
+ | 0.5467 | 7000 | 0.0194 | 0.0189 | -30.3385 | -30.5334 | -29.1280 |
433
+ | 0.6248 | 8000 | 0.0192 | 0.0188 | -30.6629 | -30.8491 | -29.4291 |
434
+ | 0.7029 | 9000 | 0.0191 | 0.0186 | -30.6934 | -30.8920 | -29.4820 |
435
+ | 0.7810 | 10000 | 0.019 | 0.0185 | -31.0134 | -31.2205 | -29.6545 |
436
+ | 0.8591 | 11000 | 0.0189 | 0.0185 | -31.0993 | -31.2950 | -29.8062 |
437
+ | 0.9372 | 12000 | 0.0188 | 0.0184 | -31.0707 | -31.2847 | -29.7483 |
438
+
439
+
440
+ ### Framework Versions
441
+ - Python: 3.10.12
442
+ - Sentence Transformers: 3.3.1
443
+ - Transformers: 4.46.3
444
+ - PyTorch: 2.5.1+cu121
445
+ - Accelerate: 1.1.1
446
+ - Datasets: 3.1.0
447
+ - Tokenizers: 0.20.3
448
+
449
+ ## Citation
450
+
451
+ ### BibTeX
452
+
453
+ #### Sentence Transformers
454
+ ```bibtex
455
+ @inproceedings{reimers-2019-sentence-bert,
456
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
457
+ author = "Reimers, Nils and Gurevych, Iryna",
458
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
459
+ month = "11",
460
+ year = "2019",
461
+ publisher = "Association for Computational Linguistics",
462
+ url = "https://arxiv.org/abs/1908.10084",
463
+ }
464
+ ```
465
+
466
+ <!--
467
+ ## Glossary
468
+
469
+ *Clearly define terms in order to be accessible across audiences.*
470
+ -->
471
+
472
+ <!--
473
+ ## Model Card Authors
474
+
475
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
476
+ -->
477
+
478
+ <!--
479
+ ## Model Card Contact
480
+
481
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
482
+ -->
config.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "google-bert/bert-base-multilingual-cased",
3
+ "architectures": [
4
+ "BertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "directionality": "bidi",
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-12,
15
+ "max_position_embeddings": 512,
16
+ "model_type": "bert",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 0,
20
+ "pooler_fc_size": 768,
21
+ "pooler_num_attention_heads": 12,
22
+ "pooler_num_fc_layers": 3,
23
+ "pooler_size_per_head": 128,
24
+ "pooler_type": "first_token_transform",
25
+ "position_embedding_type": "absolute",
26
+ "torch_dtype": "float32",
27
+ "transformers_version": "4.46.3",
28
+ "type_vocab_size": 2,
29
+ "use_cache": true,
30
+ "vocab_size": 119547
31
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.3.1",
4
+ "transformers": "4.46.3",
5
+ "pytorch": "2.5.1+cu121"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": "cosine"
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3c3e5a1477bfcb62c0a33a522b96a612b37034f413d06396b2dc35d0ba98a12
3
+ size 711436136
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 128,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": false,
47
+ "mask_token": "[MASK]",
48
+ "model_max_length": 512,
49
+ "pad_token": "[PAD]",
50
+ "sep_token": "[SEP]",
51
+ "strip_accents": null,
52
+ "tokenize_chinese_chars": true,
53
+ "tokenizer_class": "BertTokenizer",
54
+ "unk_token": "[UNK]"
55
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff