giladgd commited on
Commit
dfad7d6
·
verified ·
1 Parent(s): b5b1cdd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -5
README.md CHANGED
@@ -29,7 +29,6 @@ Static quants of [`Qwen/Qwen3-Reranker-0.6B`](https://huggingface.co/Qwen/Qwen3-
29
  | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q5_K_M.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q5_K_M` | Q5_K_M | 444.2MB |
30
  | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q6_K.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q6_K` | Q6_K | 494.9MB |
31
  | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q8_0.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0` | Q8_0 | 639.2MB |
32
- | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.MXFP4.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:MXFP4` | MXFP4 | 639.2MB |
33
  | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.F16.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:F16` | F16 | 1.2GB |
34
 
35
  > [!TIP]
@@ -49,7 +48,7 @@ brew install nodejs
49
  ### CLI
50
  Chat with the model:
51
  ```bash
52
- npx -y node-llama-cpp chat hf:giladgd/Qwen3-Reranker-0.6B-GGUF:MXFP4
53
  ```
54
 
55
  ### Code
@@ -61,7 +60,7 @@ npm install node-llama-cpp
61
  ```typescript
62
  import {getLlama, resolveModelFile, LlamaChatSession} from "node-llama-cpp";
63
 
64
- const modelUri = "hf:giladgd/Qwen3-Reranker-0.6B-GGUF:MXFP4";
65
 
66
 
67
  const llama = await getLlama();
@@ -94,10 +93,10 @@ brew install llama.cpp
94
 
95
  ### CLI
96
  ```bash
97
- llama-cli -hf giladgd/Qwen3-Reranker-0.6B-GGUF:MXFP4 -p "The meaning to life and the universe is"
98
  ```
99
 
100
  ### Server
101
  ```bash
102
- llama-server -hf giladgd/Qwen3-Reranker-0.6B-GGUF:MXFP4 -c 2048
103
  ```
 
29
  | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q5_K_M.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q5_K_M` | Q5_K_M | 444.2MB |
30
  | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q6_K.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q6_K` | Q6_K | 494.9MB |
31
  | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.Q8_0.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0` | Q8_0 | 639.2MB |
 
32
  | [GGUF](https://huggingface.co/giladgd/Qwen3-Reranker-0.6B-GGUF/resolve/main/Qwen3-Reranker-0.6B.F16.gguf) | `hf:giladgd/Qwen3-Reranker-0.6B-GGUF:F16` | F16 | 1.2GB |
33
 
34
  > [!TIP]
 
48
  ### CLI
49
  Chat with the model:
50
  ```bash
51
+ npx -y node-llama-cpp chat hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0
52
  ```
53
 
54
  ### Code
 
60
  ```typescript
61
  import {getLlama, resolveModelFile, LlamaChatSession} from "node-llama-cpp";
62
 
63
+ const modelUri = "hf:giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0";
64
 
65
 
66
  const llama = await getLlama();
 
93
 
94
  ### CLI
95
  ```bash
96
+ llama-cli -hf giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0 -p "The meaning to life and the universe is"
97
  ```
98
 
99
  ### Server
100
  ```bash
101
+ llama-server -hf giladgd/Qwen3-Reranker-0.6B-GGUF:Q8_0 -c 2048
102
  ```