Update README.md

README.md

@@ -169,20 +169,26 @@ response = generate(model, tokenizer, prompt=prompt, max_tokens=400)
 
 Apache 2.0
 
+#### Original model card below:
+
 ------
 
-# Original model Card: palmyra-mini-thinking-a
 
-## Model Details
+<div align="center">
+  <h1>Palmyra-mini-thinking-a</h1>
 
-**Model Name:** palmyra-mini-thinking-a
+</div>
 
-**Version:** 1.0
+### Model Description
 
-**Type:** Generative AI Language Model
+- **Language(s) (NLP):** English
+- **License:** Apache-2.0
+- **Finetuned from model:** Qwen/Qwen2.5-1.5B
+- **Context window:** 131,072 tokens
+- **Parameters:** 1.7 billion
 
 
-## Model Description
+## Model Details
 
 The palmyra-mini-thinking-a model demonstrates exceptional performance in advanced mathematical reasoning and competitive programming. Its capabilities are highlighted by an outstanding score of 0.886 on the 'MATH500' benchmark, showcasing a robust ability to solve complex mathematical problems. The strength of the model in quantitative challenges is further confirmed by its score of 0.8287 on 'gsm8k (strict-match)', which demonstrates proficiency in multi-step arithmetic reasoning. Additionally, the model proves its aptitude for high-level problem-solving with a score of 0.8 on 'AMC23'. The model also shows strong potential in the coding domain, achieving a score of 0.5631 on 'Codeforces (pass_rate)' and 0.5481 on 'Olympiadbench (extractive_match)', indicating competence in generating correct solutions for programming challenges.
 
@@ -217,14 +223,88 @@ This section provides a detailed breakdown of the palmyra-mini-thinking-a model'
 | HMMT23 (extractive_match)                                        | 0.1      |
 | Average                                                          | 0.380839 |
 
-## Intended Use
 
-This model is intended for research and development in the field of generative AI, particularly for tasks requiring mathematical and logical reasoning.
 
-## Limitations
+### Use with transformers
+
+You can run conversational inference using the Transformers Auto classes with the `generate()` function. Here's an example:
+
+```py
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_id = "Writer/palmyra-mini-thinking-a"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.float16,
+    device_map="auto",
+    attn_implementation="flash_attention_2",
+)
+
+messages = [
+      {
+        "role": "user",
+        "content": "You have a 3-liter jug and a 5-liter jug. How can you measure exactly 4 liters of water?"
+      }
+    ],
+
+input_ids = tokenizer.apply_chat_template(
+    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
+)
+
+gen_conf = {
+    "max_new_tokens": 256,
+    "eos_token_id": tokenizer.eos_token_id,
+    "temperature": 0.3,
+    "top_p": 0.9,
+}
+
+with torch.inference_mode():
+    output_id = model.generate(input_ids, **gen_conf)
+
+output_text = tokenizer.decode(output_id[0][input_ids.shape[1] :])
+
+print(output_text)
+```
+
+## Running with vLLM
+```py
+vllm serve Writer/palmyra-mini-thinking-a
+```
+```py
+curl -X POST http://localhost:8000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "Writer/palmyra-mini-thinking-a",
+    "messages": [
+      {
+        "role": "user",
+        "content": "You have a 3-liter jug and a 5-liter jug. How can you measure exactly 4 liters of water?"
+      }
+    ],
+    "max_tokens": 8000,
+    "temperature": 0.2
+  }'
+```
 
-The model's performance has been evaluated on a specific set of benchmarks. Its performance on other tasks or in real-world applications may vary.
 
 ## Ethical Considerations
 
-As with any language model, there is a potential for generating biased or inaccurate information. Users should be aware of these limitations and use the model responsibly.
+As with any language model, there is a potential for generating biased or inaccurate information. Users should be aware of these limitations and use the model responsibly.
+
+### Citation and Related Information
+
+To cite this model:
+```
+@misc{Palmyra-mini-thinking-a,
+  author = {Writer Engineering team},
+  title = {{Palmyra-mini: A powerful LLM designed for math and coding}},
+  howpublished = {\url{https://dev.writer.com}},
+  year = 2025,
+  month = Sep 
+}
+```
+Contact Hello@writer.com