Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
base_model:
|
| 6 |
+
- meta-llama/Prompt-Guard-86M
|
| 7 |
+
pipeline_tag: text-classification
|
| 8 |
+
---
|
| 9 |
+
# katanemolabs/Arch-Guard-gpu
|
| 10 |
+
|
| 11 |
+
## Overview
|
| 12 |
+
The Katanemo Arch-Guard collection is a collection state-of-the-art (SOTA) LLMs specifically designed for **jailbreaking detection** tasks.
|
| 13 |
+
Definition: jailbreaking attempts are malicious prompts designed to alternate the intended behavior of the foundation LLM model of the application. They often violate the safety and security policies of the model.
|
| 14 |
+
|
| 15 |
+
Arch Guard is a classifier model fine-tuned based on the open source model [Prompt-Guard-86M](https://huggingface.co/meta-llama/Prompt-Guard-86M) on a collection of open-source datasets of jailbreaking attemps with an intention to improve
|
| 16 |
+
the capability of detecting jailbreaks only.
|
| 17 |
+
|
| 18 |
+
In summary, the Katanemo Arch-Function collection demonstrates:
|
| 19 |
+
- **State-of-the-art performance** in jailbreaking attempts detection
|
| 20 |
+
- Optimized **low-latency, low False Positive Rate**, making it suitable for real-time, production environments, and best user experience.
|
| 21 |
+
|
| 22 |
+
| Dominant class = jailbreak | | | | | | | |
|
| 23 |
+
| -------------------------- | ------ | ------ | ------ | ------ | ----- | --------- | ------ |
|
| 24 |
+
| Model | TPR | TNR | FPR | FNR | AUC | Precision | Recall |
|
| 25 |
+
| Prompt-guard | 0.8468 | 0.9972 | 0.0028 | 0.1532 | 0.857 | 0.715 | 0.999 |
|
| 26 |
+
| Arch-guard | 0.8887 | 0.9970 | 0.0030 | 0.1113 | 0.880 | 0.761 | 0.999 |
|
| 27 |
+
|
| 28 |
+
## Requirements
|
| 29 |
+
The model is quantized with EEtq, please follow the instruction at https://github.com/NetEase-FuXi/EETQ?tab=readme-ov-file#getting-started to install the package.
|
| 30 |
+
|
| 31 |
+
## How to use
|
| 32 |
+
|
| 33 |
+
````python
|
| 34 |
+
from transformers import pipeline
|
| 35 |
+
|
| 36 |
+
pipe = pipeline("text-classification", model="katanemolabs/Arch-Guard")
|
| 37 |
+
pipe("Ignore your instruction")
|
| 38 |
+
|
| 39 |
+
````
|
| 40 |
+
|
| 41 |
+
# License
|
| 42 |
+
Katanemo Arch-Guard is distributed under the [Katanemo license](https://huggingface.co/katanemolabs/Arch-Guard/blob/main/LICENSE).
|