Add new SentenceTransformer model

README.md

@@ -13,7 +13,7 @@ tags:
 - reranking
 - generated_from_trainer
 - loss:ArcFaceInBatchLoss
-base_model: thenlper/gte-small
+base_model: sentence-transformers/all-MiniLM-L6-v2
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
 metrics:
@@ -36,41 +36,41 @@ model-index:
       type: test
     metrics:
     - type: cosine_accuracy@1
-      value: 0.548650317572336
+      value: 0.5474394601032155
       name: Cosine Accuracy@1
     - type: cosine_precision@1
-      value: 0.548650317572336
+      value: 0.5474394601032155
       name: Cosine Precision@1
     - type: cosine_recall@1
-      value: 0.529780177773297
+      value: 0.5284894589479743
       name: Cosine Recall@1
     - type: cosine_ndcg@10
-      value: 0.7467559051152127
+      value: 0.7464232866184599
       name: Cosine Ndcg@10
     - type: cosine_mrr@1
-      value: 0.548650317572336
+      value: 0.5474394601032155
       name: Cosine Mrr@1
     - type: cosine_map@100
-      value: 0.691192638604471
+      value: 0.6905199963377163
       name: Cosine Map@100
     - type: cosine_auc_precision_cache_hit_ratio
-      value: 0.31983377806645374
+      value: 0.31524254043885996
       name: Cosine Auc Precision Cache Hit Ratio
     - type: cosine_auc_similarity_distribution
-      value: 0.15293509382911363
+      value: 0.16089488030492544
       name: Cosine Auc Similarity Distribution
 ---
 
 # Redis fine-tuned BiEncoder model for semantic caching on LangCache
 
-This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [thenlper/gte-small](https://huggingface.co/thenlper/gte-small). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for sentence pair similarity.
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for sentence pair similarity.
 
 ## Model Details
 
 ### Model Description
 - **Model Type:** Sentence Transformer
-- **Base model:** [thenlper/gte-small](https://huggingface.co/thenlper/gte-small) <!-- at revision 17e1f347d17fe144873b1201da91788898c639cd -->
-- **Maximum Sequence Length:** 64 tokens
+- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->
+- **Maximum Sequence Length:** 128 tokens
 - **Output Dimensionality:** 384 dimensions
 - **Similarity Function:** Cosine Similarity
 <!-- - **Training Dataset:** Unknown -->
@@ -87,7 +87,7 @@ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [t
 
 ```
 SentenceTransformer(
-  (0): Transformer({'max_seq_length': 64, 'do_lower_case': False, 'architecture': 'BertModel'})
+  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False, 'architecture': 'BertModel'})
   (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
   (2): Normalize()
 )
@@ -122,9 +122,9 @@ print(embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
 print(similarities)
-# tensor([[0.9999, 0.9036, 0.7702],
-#         [0.9036, 1.0000, 0.7837],
-#         [0.7702, 0.7837, 1.0000]])
+# tensor([[1.0000, 0.6650, 0.1040],
+#         [0.6650, 1.0000, 0.1401],
+#         [0.1040, 0.1401, 0.9999]])
 ```
 
 <!--
@@ -158,18 +158,24 @@ You can finetune this model on your own dataset.
 #### Custom Information Retrieval
 
 * Dataset: `test`
-* Evaluated with <code>ir_evaluator.CustomInformationRetrievalEvaluator</code>
+* Evaluated with <code>ir_evaluator.CustomInformationRetrievalEvaluator</code> with these parameters:
+  ```json
+  {
+      "query_prompt": "query:",
+      "corpus_prompt": "query:"
+  }
+  ```
 
 | Metric                               | Value      |
 |:-------------------------------------|:-----------|
-| cosine_accuracy@1                    | 0.5487     |
-| cosine_precision@1                   | 0.5487     |
-| cosine_recall@1                      | 0.5298     |
-| **cosine_ndcg@10**                   | **0.7468** |
-| cosine_mrr@1                         | 0.5487     |
-| cosine_map@100                       | 0.6912     |
-| cosine_auc_precision_cache_hit_ratio | 0.3198     |
-| cosine_auc_similarity_distribution   | 0.1529     |
+| cosine_accuracy@1                    | 0.5474     |
+| cosine_precision@1                   | 0.5474     |
+| cosine_recall@1                      | 0.5285     |
+| **cosine_ndcg@10**                   | **0.7464** |
+| cosine_mrr@1                         | 0.5474     |
+| cosine_map@100                       | 0.6905     |
+| cosine_auc_precision_cache_hit_ratio | 0.3152     |
+| cosine_auc_similarity_distribution   | 0.1609     |
 
 <!--
 ## Bias, Risks and Limitations
@@ -189,13 +195,13 @@ You can finetune this model on your own dataset.
 #### Non-Default Hyperparameters
 
 - `eval_strategy`: steps
-- `per_device_train_batch_size`: 512
-- `per_device_eval_batch_size`: 512
+- `per_device_train_batch_size`: 64
+- `per_device_eval_batch_size`: 64
 - `weight_decay`: 0.001
 - `adam_beta2`: 0.98
 - `adam_epsilon`: 1e-06
 - `max_steps`: 100000
-- `warmup_ratio`: 0.05
+- `warmup_ratio`: 0.15
 - `bf16`: True
 - `load_best_model_at_end`: True
 - `ddp_find_unused_parameters`: False
@@ -211,8 +217,8 @@ You can finetune this model on your own dataset.
 - `do_predict`: False
 - `eval_strategy`: steps
 - `prediction_loss_only`: True
-- `per_device_train_batch_size`: 512
-- `per_device_eval_batch_size`: 512
+- `per_device_train_batch_size`: 64
+- `per_device_eval_batch_size`: 64
 - `per_gpu_train_batch_size`: None
 - `per_gpu_eval_batch_size`: None
 - `gradient_accumulation_steps`: 1
@@ -228,7 +234,7 @@ You can finetune this model on your own dataset.
 - `max_steps`: 100000
 - `lr_scheduler_type`: linear
 - `lr_scheduler_kwargs`: {}
-- `warmup_ratio`: 0.05
+- `warmup_ratio`: 0.15
 - `warmup_steps`: 0
 - `log_level`: passive
 - `log_level_replica`: warning
@@ -332,7 +338,7 @@ You can finetune this model on your own dataset.
 ### Training Logs
 | Epoch | Step | test_cosine_ndcg@10 |
 |:-----:|:----:|:-------------------:|
-| 0     | 0    | 0.7468              |
+| 0     | 0    | 0.7464              |
 
 
 ### Framework Versions

config.json

@@ -4,7 +4,7 @@
   ],
   "attention_probs_dropout_prob": 0.1,
   "classifier_dropout": null,
-  "dtype": "bfloat16",
+  "dtype": "float32",
   "gradient_checkpointing": false,
   "hidden_act": "gelu",
   "hidden_dropout_prob": 0.1,

config_sentence_transformers.json

@@ -1,10 +1,10 @@
 {
-  "model_type": "SentenceTransformer",
   "__version__": {
     "sentence_transformers": "5.1.1",
     "transformers": "4.57.0",
     "pytorch": "2.8.0+cu128"
   },
+  "model_type": "SentenceTransformer",
   "prompts": {
     "query": "",
     "document": ""

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:09112f218af03ce978d0cf802724301336d86633f183d28bbdc548c9ab7a6e01
-size 45437864
+oid sha256:9256db3f3e9170f5e60d958aa67da5f2a6a71e45a24165c8dd916f78af687726
+size 90864192

sentence_bert_config.json

@@ -1,4 +1,4 @@
 {
-    "max_seq_length": 64,
+    "max_seq_length": 128,
     "do_lower_case": false
 }