Update safetensors keys; update README usage

README.md

@@ -3,10 +3,13 @@ language: en
 license: apache-2.0
 tags:
 - sentence-transformers
+- transformers
 - opinion-mining
 - stance-detection
 - social-computing
 - LoRA
+base_model: sentence-transformers/all-mpnet-base-v2
+library_name: peft
 ---
 
 # Stance-Aware Sentence Transformers for Opinion Mining
@@ -23,43 +26,72 @@ As explained in our [EMNLP 2024 paper](https://scholar.google.com/citations?view
 
 ### Loading and Using the Model with LoRA Weights
 
-Since this model leverages LoRA (Low-Rank Adaptation) fine-tuning, you only need to download the lightweight LoRA weights and apply them to the base model. Below is a step-by-step guide on loading and applying the LoRA weights to `sentence-transformers/all-mpnet-base-v2`.
+Since this model leverages LoRA (Low-Rank Adaptation) fine-tuning, you only need to download the lightweight LoRA weights and apply them to the base model. Below are two guides on loading and applying the LoRA weights to `sentence-transformers/all-mpnet-base-v2` based on Sentence Transformers and Transformers, respectively.
 
-#### Requirements
-First, ensure you have the required libraries installed:
+#### with Sentence Transformers
 
 ```bash
-!pip install peft transformers sentence-transformers torch
-
-from peft import LoraConfig, get_peft_model, prepare_model_for_int8_training, TaskType, PeftModel # peft-0.7.1
-from transformers import (
-    AutoModel,
-    AutoTokenizer,
-    BitsAndBytesConfig,
-    HfArgumentParser,
-    AutoTokenizer,
-    TrainingArguments,
-    AutoConfig,
-)
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
+pip install peft sentence-transformers
 ```
 
+```python
+from sentence_transformers import SentenceTransformer
+from peft import PeftModel
 
+model = SentenceTransformer("all-mpnet-base-v2")
+model[0].auto_model = PeftModel.from_pretrained(model[0].auto_model, "vahidthegreat/StanceAware-SBERT")
 
-## Using the Model with the Siamese Network Class
+sentences = [
+    "I love pineapple on pizza",
+    "I hate pineapple on pizza",
+    "I like pineapple on pizza",
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+
+similarity = model.similarity(embeddings, embeddings)
+print(similarity)
+```
+
+```output
+(3, 768)
+tensor([[1.0000, 0.5732, 0.9713],
+        [0.5732, 1.0000, 0.5804],
+        [0.9713, 0.5804, 1.0000]])
+```
+From these results we can see that the first and third sentence are very similar, while the second sentence is less similar to the other two.
+
+#### with Transformers
+First, ensure you have the required libraries installed:
 
-The following custom `SiameseNetworkMPNet` class leverages the model for stance detection tasks. It pools embeddings and normalizes them for similarity calculations. This is for the sake of replicability of our exact results. But the model would work without this as well.
 ```bash
+pip install peft transformers torch
+```
+
+#### Using the Model with the Siamese Network Class
+
+The following custom `SiameseNetworkMPNet` class leverages the model for stance detection tasks. It pools embeddings and normalizes them for similarity calculations. This is for the sake of replicability of our exact results. But the model would work without this as well.
+
+```python
+from peft import PeftModel
+from transformers import AutoModel, AutoTokenizer
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
 class SiameseNetworkMPNet(nn.Module):
     def __init__(self, model_name, tokenizer, normalize=True):
         super(SiameseNetworkMPNet, self).__init__()
 
-        self.model = AutoModel.from_pretrained(model_name)#, quantization_config=bnb_config, trust_remote_code=True)
+        self.model = AutoModel.from_pretrained(model_name)
         self.normalize = normalize
         self.tokenizer = tokenizer
 
+    def apply_lora_weights(self, lora_model):
+        self.model = PeftModel.from_pretrained(self.model, lora_model)
+        self.model = self.model.merge_and_unload()
+        return self
+
     def forward(self, **inputs):
         model_output = self.model(**inputs)
         attention_mask = inputs['attention_mask']
@@ -80,18 +112,13 @@ base_model = SiameseNetworkMPNet(model_name=base_model_name, tokenizer=tokenizer
 
 # Load and apply LoRA weights
 lora_model = SiameseNetworkMPNet(model_name=base_model_name, tokenizer=tokenizer)
-lora_model = PeftModel.from_pretrained(lora_model, "vahidthegreat/StanceAware-SBERT")
-lora_model = lora_model.merge_and_unload()
-
-base_model.eval()
-lora_model.eval()
+lora_model.apply_lora_weights("vahidthegreat/StanceAware-SBERT")
 ```
 
-
-## Example Usage for Two-Sentence Similarity
+#### Example Usage for Two-Sentence Similarity
 The following example shows how to use the Siamese network with two input sentences, calculating cosine similarity to compare stances.
 
-```bash
+```python
 from sklearn.metrics.pairwise import cosine_similarity
 
 def two_sentence_similarity(model, tokenizer, text1, text2):

adapter_config.json

@@ -1,10 +1,10 @@
 {
   "alpha_pattern": {},
   "auto_mapping": {
-    "base_model_class": "SiameseNetworkMPNet",
-    "parent_library": "__main__"
+    "base_model_class": "MPNetModel",
+    "parent_library": "transformers.models.mpnet.modeling_mpnet"
   },
-  "base_model_name_or_path": null,
+  "base_model_name_or_path": "sentence-transformers/all-mpnet-base-v2",
   "bias": "none",
   "fan_in_fan_out": false,
   "inference_mode": true,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:88489d3bfc6549f00b19729364235e1d6bbdfce32d53e469c152d4eac48624f8
-size 9450464
+oid sha256:bb14ba34a5419b21cc4abf7bd50f4d3d4c009ef36bd9c95640a251e49d3fca9e
+size 9449888