rufimelo commited on
Commit
deca15e
1 Parent(s): 05751e7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +54 -48
README.md CHANGED
@@ -1,17 +1,47 @@
 
1
  ---
 
 
 
2
  pipeline_tag: sentence-similarity
3
  tags:
4
  - sentence-transformers
5
- - feature-extraction
6
  - sentence-similarity
7
  - transformers
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  ---
9
 
10
- # {MODEL_NAME}
11
 
12
  This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 1024 dimensional dense vector space and can be used for tasks like clustering or semantic search.
 
13
 
14
- <!--- Describe your model here -->
15
 
16
  ## Usage (Sentence-Transformers)
17
 
@@ -25,9 +55,9 @@ Then you can use the model like this:
25
 
26
  ```python
27
  from sentence_transformers import SentenceTransformer
28
- sentences = ["This is an example sentence", "Each sentence is converted"]
29
 
30
- model = SentenceTransformer('{MODEL_NAME}')
31
  embeddings = model.encode(sentences)
32
  print(embeddings)
33
  ```
@@ -35,7 +65,7 @@ print(embeddings)
35
 
36
 
37
  ## Usage (HuggingFace Transformers)
38
- Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
39
 
40
  ```python
41
  from transformers import AutoTokenizer, AutoModel
@@ -53,8 +83,8 @@ def mean_pooling(model_output, attention_mask):
53
  sentences = ['This is an example sentence', 'Each sentence is converted']
54
 
55
  # Load model from HuggingFace Hub
56
- tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
57
- model = AutoModel.from_pretrained('{MODEL_NAME}')
58
 
59
  # Tokenize sentences
60
  encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
@@ -70,56 +100,32 @@ print("Sentence embeddings:")
70
  print(sentence_embeddings)
71
  ```
72
 
73
-
74
-
75
- ## Evaluation Results
76
-
77
- <!--- Describe how your model was evaluated -->
78
-
79
- For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
80
-
81
-
82
  ## Training
83
- The model was trained with the parameters:
84
-
85
- **DataLoader**:
86
 
87
- `torch.utils.data.dataloader.DataLoader` of length 2157 with parameters:
88
- ```
89
- {'batch_size': 8, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
90
- ```
91
 
92
- **Loss**:
93
-
94
- `sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss`
95
-
96
- Parameters of the fit()-Method:
97
- ```
98
- {
99
- "epochs": 5,
100
- "evaluation_steps": 0,
101
- "evaluator": "NoneType",
102
- "max_grad_norm": 1,
103
- "optimizer_class": "<class 'transformers.optimization.AdamW'>",
104
- "optimizer_params": {
105
- "lr": 1e-05
106
- },
107
- "scheduler": "WarmupLinear",
108
- "steps_per_epoch": null,
109
- "warmup_steps": 1079,
110
- "weight_decay": 0.01
111
- }
112
- ```
113
 
114
 
115
  ## Full Model Architecture
116
  ```
117
  SentenceTransformer(
118
- (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
119
- (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
120
  )
121
  ```
122
 
123
  ## Citing & Authors
124
 
125
- <!--- Describe where people can find more information -->
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
  ---
3
+ language:
4
+ - pt
5
+ thumbnail: "Portuguese SBERT for STS"
6
  pipeline_tag: sentence-similarity
7
  tags:
8
  - sentence-transformers
 
9
  - sentence-similarity
10
  - transformers
11
+ datasets:
12
+ - assin
13
+ - assin2
14
+ - stsb_multi_mt
15
+ widget:
16
+ - source_sentence: "O advogado apresentou as provas ao juíz."
17
+ sentences:
18
+ - "O juíz leu as provas."
19
+ - "O juíz leu o recurso."
20
+ - "O juíz atirou uma pedra."
21
+ example_title: "Example 1"
22
+ model-index:
23
+ - name: BERTimbau
24
+ results:
25
+ - task:
26
+ name: STS
27
+ type: STS
28
+ metrics:
29
+ - name: Pearson Correlation - assin Dataset
30
+ type: Pearson Correlation
31
+ value: xxxxxx
32
+ - name: Pearson Correlation - assin2 Dataset
33
+ type: Pearson Correlation
34
+ value: xxxxxx
35
+ - name: Pearson Correlation - stsb_multi_mt pt Dataset
36
+ type: Pearson Correlation
37
+ value: xxxxxx
38
  ---
39
 
40
+ # rufimelo/bert-large-portuguese-cased-sts2
41
 
42
  This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 1024 dimensional dense vector space and can be used for tasks like clustering or semantic search.
43
+ rufimelo/bert-large-portuguese-cased-sts derives from [BERTimbau](https://huggingface.co/neuralmind/bert-large-portuguese-cased) large.
44
 
 
45
 
46
  ## Usage (Sentence-Transformers)
47
 
 
55
 
56
  ```python
57
  from sentence_transformers import SentenceTransformer
58
+ sentences = ["Isto é um exemplo", "Isto é um outro exemplo"]
59
 
60
+ model = SentenceTransformer('rufimelo/Legal-BERTimbau-sts-large-v2')
61
  embeddings = model.encode(sentences)
62
  print(embeddings)
63
  ```
 
65
 
66
 
67
  ## Usage (HuggingFace Transformers)
68
+
69
 
70
  ```python
71
  from transformers import AutoTokenizer, AutoModel
 
83
  sentences = ['This is an example sentence', 'Each sentence is converted']
84
 
85
  # Load model from HuggingFace Hub
86
+ tokenizer = AutoTokenizer.from_pretrained('rufimelo/bert-large-portuguese-cased-sts')
87
+ model = AutoModel.from_pretrained('rufimelo/bert-large-portuguese-cased-sts')
88
 
89
  # Tokenize sentences
90
  encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
 
100
  print(sentence_embeddings)
101
  ```
102
 
 
 
 
 
 
 
 
 
 
103
  ## Training
 
 
 
104
 
105
+ rufimelo/bert-large-portuguese-cased-sts derives from [BERTimbau](https://huggingface.co/neuralmind/bert-base-portuguese-cased) large.
 
 
 
106
 
107
+ It was trained for Semantic Textual Similarity, being submitted to a fine tuning stage with the [assin](https://huggingface.co/datasets/assin), [assin2](https://huggingface.co/datasets/assin2) and [stsb_multi_mt pt](https://huggingface.co/datasets/stsb_multi_mt) datasets.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
108
 
109
 
110
  ## Full Model Architecture
111
  ```
112
  SentenceTransformer(
113
+ (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel
114
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False})
115
  )
116
  ```
117
 
118
  ## Citing & Authors
119
 
120
+ If you use this work, please cite BERTimbau's work:
121
+
122
+ ```bibtex
123
+ @inproceedings{souza2020bertimbau,
124
+ author = {F{\'a}bio Souza and
125
+ Rodrigo Nogueira and
126
+ Roberto Lotufo},
127
+ title = {{BERT}imbau: pretrained {BERT} models for {B}razilian {P}ortuguese},
128
+ booktitle = {9th Brazilian Conference on Intelligent Systems, {BRACIS}, Rio Grande do Sul, Brazil, October 20-23 (to appear)},
129
+ year = {2020}
130
+ }
131
+ ```