update readme

README.md

@@ -1,5 +1,5 @@
 ---
-title: ' ZeroGPU-LLM-Inference'
+title: 'ZeroGPU-LLM-Inference'
 emoji: 🧠
 colorFrom: pink
 colorTo: purple
@@ -8,35 +8,73 @@ sdk_version: 5.25.2
 app_file: app.py
 pinned: false
 license: apache-2.0
-short_description: Chat inference for GGUF models with llama.cpp & Gradio
+short_description: Streaming LLM chat with web search and debug
 ---
 
-This Gradio app enables **chat-based inference** on various GGUF models using `llama.cpp` and `llama-cpp-python`. The application features:
-
-- **Real-Time Web Search Integration:** Uses DuckDuckGo to retrieve up-to-date context; debug output is displayed in real time.
-- **Streaming Token-by-Token Responses:** Users see the generated answer as it comes in.
-- **Response Cancellation:** A cancel button allows stopping response generation in progress.
-- **Customizable Prompts & Generation Parameters:** Adjust the system prompt (with dynamic date insertion), temperature, token limits, and more.
-- **Memory-Safe Design:** Loads one model at a time with proper memory management, ideal for deployment on Hugging Face Spaces.
-- **Rate Limit Handling:** Implements exponential backoff to cope with DuckDuckGo API rate limits.
-
-### 🔄 Supported Models:
-- `Qwen/Qwen2.5-7B-Instruct-GGUF` → `qwen2.5-7b-instruct-q2_k.gguf`
-- `unsloth/gemma-3-4b-it-GGUF` → `gemma-3-4b-it-Q4_K_M.gguf`
-- `unsloth/Phi-4-mini-instruct-GGUF` → `Phi-4-mini-instruct-Q4_K_M.gguf`
-- `MaziyarPanahi/Meta-Llama-3.1-8B-Instruct-GGUF` → `Meta-Llama-3.1-8B-Instruct.Q2_K.gguf`
-- `unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF` → `DeepSeek-R1-Distill-Llama-8B-Q2_K.gguf`
-- `MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF` → `Mistral-7B-Instruct-v0.3.IQ3_XS.gguf`
-- `Qwen/Qwen2.5-Coder-7B-Instruct-GGUF` → `qwen2.5-coder-7b-instruct-q2_k.gguf`
-
-### ⚙️ Features:
-- **Model Selection:** Select from multiple GGUF models.
-- **Customizable Prompts & Parameters:** Set a system prompt (e.g., automatically including today’s date), adjust temperature, token limits, and more.
-- **Chat-style Interface:** Interactive Gradio UI with streaming token-by-token responses.
-- **Real-Time Web Search & Debug Output:** Leverages DuckDuckGo to fetch recent context, with a dedicated debug panel showing web search progress and results.
-- **Response Cancellation:** Cancel in-progress answer generation using a cancel button.
-- **Memory-Safe & Rate-Limit Resilient:** Loads one model at a time with proper cleanup and incorporates exponential backoff to handle API rate limits.
-
-Ideal for deploying multiple GGUF chat models on Hugging Face Spaces with a robust, user-friendly interface!
-
-For further details, check the [Spaces configuration guide](https://huggingface.co/docs/hub/spaces-config-reference).
+This Gradio app provides **token-streaming, chat-style inference** on a wide variety of Transformer models—leveraging ZeroGPU for free GPU acceleration on HF Spaces.
+
+Key features:
+- **Real-time DuckDuckGo web search** (background thread, configurable timeout) with results injected into the system prompt.
+- **Prompt preview panel** for debugging and prompt-engineering insights—see exactly what’s sent to the model.
+- **Thought vs. Answer streaming**: any `<think>…</think>` blocks emitted by the model are shown as separate “💭 Thought.”
+- **Cancel button** to immediately stop generation.
+- **Dynamic system prompt**: automatically inserts today’s date when you toggle web search.
+- **Extensive model selection**: over 30 LLMs (from Phi-4 mini to Qwen3-14B, SmolLM2, Taiwan-ELM, Mistral, Meta-Llama, MiMo, Gemma, DeepSeek-R1, etc.).
+- **Memory-safe design**: loads one model at a time, clears cache after each generation.
+- **Customizable generation parameters**: max tokens, temperature, top-k, top-p, repetition penalty.
+- **Web-search settings**: max results, max chars per result, search timeout.
+- **Requirements pinned** to ensure reproducible deployment.
+
+## 🔄 Supported Models
+
+Use the dropdown to select any of these:
+
+| Name                                  | Repo ID                                            |
+| ------------------------------------- | -------------------------------------------------- |
+| Taiwan-ELM-1_1B-Instruct              | liswei/Taiwan-ELM-1_1B-Instruct                    |
+| Taiwan-ELM-270M-Instruct              | liswei/Taiwan-ELM-270M-Instruct                    |
+| Qwen3-0.6B                            | Qwen/Qwen3-0.6B                                    |
+| Qwen3-1.7B                            | Qwen/Qwen3-1.7B                                    |
+| Qwen3-4B                              | Qwen/Qwen3-4B                                      |
+| Qwen3-8B                              | Qwen/Qwen3-8B                                      |
+| Qwen3-14B                             | Qwen/Qwen3-14B                                     |
+| Gemma-3-4B-IT                         | unsloth/gemma-3-4b-it                              |
+| SmolLM2-135M-Instruct-TaiwanChat      | Luigi/SmolLM2-135M-Instruct-TaiwanChat             |
+| SmolLM2-135M-Instruct                 | HuggingFaceTB/SmolLM2-135M-Instruct                |
+| SmolLM2-360M-Instruct-TaiwanChat      | Luigi/SmolLM2-360M-Instruct-TaiwanChat             |
+| Llama-3.2-Taiwan-3B-Instruct          | lianghsun/Llama-3.2-Taiwan-3B-Instruct             |
+| MiniCPM3-4B                           | openbmb/MiniCPM3-4B                                |
+| Qwen2.5-3B-Instruct                   | Qwen/Qwen2.5-3B-Instruct                           |
+| Qwen2.5-7B-Instruct                   | Qwen/Qwen2.5-7B-Instruct                           |
+| Phi-4-mini-Reasoning                  | microsoft/Phi-4-mini-reasoning                     |
+| Phi-4-mini-Instruct                   | microsoft/Phi-4-mini-instruct                      |
+| Meta-Llama-3.1-8B-Instruct            | MaziyarPanahi/Meta-Llama-3.1-8B-Instruct            |
+| DeepSeek-R1-Distill-Llama-8B          | unsloth/DeepSeek-R1-Distill-Llama-8B               |
+| Mistral-7B-Instruct-v0.3              | MaziyarPanahi/Mistral-7B-Instruct-v0.3              |
+| Qwen2.5-Coder-7B-Instruct             | Qwen/Qwen2.5-Coder-7B-Instruct                     |
+| Qwen2.5-Omni-3B                       | Qwen/Qwen2.5-Omni-3B                               |
+| MiMo-7B-RL                            | XiaomiMiMo/MiMo-7B-RL                              |
+
+*(…and more can easily be added in `MODELS` in `app.py`.)*
+
+## ⚙️ Generation & Search Parameters
+
+- **Max Tokens**: 64–16384  
+- **Temperature**: 0.1–2.0  
+- **Top-K**: 1–100  
+- **Top-P**: 0.1–1.0  
+- **Repetition Penalty**: 1.0–2.0  
+
+- **Enable Web Search**: on/off  
+- **Max Results**: integer  
+- **Max Chars/Result**: integer  
+- **Search Timeout (s)**: 0.0–30.0  
+
+## 🚀 How It Works
+
+1. **User message** enters chat history.  
+2. If search is enabled, a background DuckDuckGo thread fetches snippets.  
+3. After up to *Search Timeout* seconds, snippets merge into the system prompt.  
+4. The selected model pipeline is loaded (bf16→f16→f32 fallback) on ZeroGPU.  
+5. Prompt is formatted—any `<think>…</think>` blocks will be streamed as separate “💭 Thought.”  
+6. Tokens stream to the Chatbot UI. Press **Cancel** to stop mid-generation.