Update README.md
Browse files
README.md
CHANGED
|
@@ -6,4 +6,14 @@ license: apache-2.0
|
|
| 6 |
|
| 7 |
# Instinct, an Open-Source Next-Edit Predictor
|
| 8 |
|
| 9 |
-
This repo contains the model weights for **Instinct**, Continue's state-of-the-art open next-edit model. Robustly fine-tuned from **Qwen2.5-Coder-7B**, Instinct intelligently predicts your next move to keep you in flow.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
|
| 7 |
# Instinct, an Open-Source Next-Edit Predictor
|
| 8 |
|
| 9 |
+
This repo contains the model weights for **Instinct**, Continue's state-of-the-art open next-edit model. Robustly fine-tuned from **Qwen2.5-Coder-7B**, Instinct intelligently predicts your next move to keep you in flow.
|
| 10 |
+
|
| 11 |
+
## Serving the model
|
| 12 |
+
|
| 13 |
+
There are many ways to serve a local model with Continue. If you wish to serve the model using [SGLang](https://github.com/sgl-project/sglang), as we did internally, you can use the following command, optionally adding `--quantization fp8` if desired.
|
| 14 |
+
|
| 15 |
+
`python3 -m sglang.launch_server --model-path continuedev/instinct --load-format safetensors`
|
| 16 |
+
|
| 17 |
+
## Learn more
|
| 18 |
+
|
| 19 |
+
For more information on the work behind Instinct, please refer to our blog.
|