Update README.md

README.md

@@ -5,12 +5,16 @@ tags:
 - feature-extraction
 - sentence-similarity
 - transformers
-
+- negation
+license: mit
+language:
+- en
 ---
 
-# {MODEL_NAME}
+# NegMPNet
 
-This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
+This is a negation-aware version of (all-mpnet-base-v2)[https://huggingface.co/sentence-transformers/all-mpnet-base-v2].
+It is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
 
 <!--- Describe your model here -->
 
@@ -28,12 +32,31 @@ Then you can use the model like this:
 from sentence_transformers import SentenceTransformer
 sentences = ["This is an example sentence", "Each sentence is converted"]
 
-model = SentenceTransformer('{MODEL_NAME}')
+model = SentenceTransformer("tum-nlp/NegMPNet")
 embeddings = model.encode(sentences)
 print(embeddings)
 ```
 
+## Negation-awareness
+This model has a better sensitivity towards negations compared to its base model. You can try it yourself:
+```python
+from sentence_transformers import SentenceTransformer, util
+import torch
+
+base_model = SentenceTransformer("sentence-transformers/all-mpnet-base-v2")
+finetuned_model = SentenceTransformer("tum-nlp/NegMPNet")
+
+def cos_similarities(references: list, candidates: list, model: SentenceTransformer, batch_size=8) -> torch.Tensor:
+    assert len(references) == len(candidates), "Number of references and candidates must be equal"
+    emb_ref = model.encode(references, batch_size=batch_size)
+    emb_cand = model.encode(candidates, batch_size=batch_size)
+    return torch.diag(util.cos_sim(emb_ref, emb_cand))
 
+references = ["Ray charles is legendary.", "Ray charles is legendary"]
+candidates = ["Ray charles is a legend.", "Ray charles isn't legendary."]
+print(cos_similarities(references, candidates, base_model)) # prints tensor([0.9453, 0.8683]) -> no negation-awareness
+print(cos_similarities(references, candidates, finetuned_model)) # prints tensor([0.9585, 0.4263]) -> sensitive to negation
+```
 
 ## Usage (HuggingFace Transformers)
 Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
@@ -54,8 +77,8 @@ def mean_pooling(model_output, attention_mask):
 sentences = ['This is an example sentence', 'Each sentence is converted']
 
 # Load model from HuggingFace Hub
-tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
-model = AutoModel.from_pretrained('{MODEL_NAME}')
+tokenizer = AutoTokenizer.from_pretrained("tum-nlp/NegMPNet")
+model = AutoModel.from_pretrained("tum-nlp/NegMPNet")
 
 # Tokenize sentences
 encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
@@ -126,4 +149,5 @@ SentenceTransformer(
 
 ## Citing & Authors
 
-<!--- Describe where people can find more information -->
+If you use our model, please cite our INLG 2023 paper:  
+tba