|
|
--- |
|
|
license: mit |
|
|
base_model: MiniMaxAI/MiniMax-M2 |
|
|
base_model_relation: quantized |
|
|
quantized_by: turboderp |
|
|
tags: |
|
|
- exl3 |
|
|
--- |
|
|
|
|
|
EXL3 quants of [MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2) |
|
|
|
|
|
⚠️ Requires ExLlamaV3 v0.0.12 (or v0.0.11 `dev` branch) |
|
|
|
|
|
Base bitrates: |
|
|
|
|
|
[2.00 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/2.0bpw) |
|
|
[3.00 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/3.0bpw) |
|
|
[4.00 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/4.0bpw) |
|
|
|
|
|
Optimized: |
|
|
|
|
|
[2.04 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/2.04bpw) |
|
|
[2.27 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/2.27bpw) |
|
|
[3.04 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/3.04bpw) |
|
|
[3.50 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/3.5bpw) |
|
|
[4.03 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/4.03bpw) |
|
|
|
|
|
|
|
|
. | KL-div | ppl | HumanEval@1 |
|
|
---------|--------|-------|------------- |
|
|
2.00 bpw | 0.400 | 10.92 | 80.5% |
|
|
2.04 bpw | 0.297 | 10.23 | 87.1% |
|
|
2.27 bpw | 0.252 | 9.78 | 88.4% |
|
|
3.00 bpw | 0.141 | 8.99 | 87.8% |
|
|
3.04 bpw | 0.117 | 8.73 | 87.2% |
|
|
3.50 bpw | 0.094 | 8.78 | 88.4% |
|
|
4.00 bpw | 0.087 | 8.58 | 89.6% |
|
|
4.03 bpw | 0.077 | 8.61 | 87.8% |
|
|
original | - | 8.51 | 87.2%¹ |
|
|
|
|
|
¹ Unconfirmed |
|
|
|
|
|
<table> |
|
|
<tr> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/2.0bpw.svg"> |
|
|
<img src="2.0bpw.svg" alt="2.00 bpw" width="160"> |
|
|
</a> |
|
|
<div>2.00 bpw</div> |
|
|
</td> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/2.04bpw.svg"> |
|
|
<img src="2.04bpw.svg" alt="2.04 bpw" width="160"> |
|
|
</a> |
|
|
<div>2.04 bpw</div> |
|
|
</td> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/2.27bpw.svg"> |
|
|
<img src="2.27bpw.svg" alt="2.27 bpw" width="160"> |
|
|
</a> |
|
|
<div>2.27 bpw</div> |
|
|
</td> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/3.0bpw.svg"> |
|
|
<img src="3.0bpw.svg" alt="3.00 bpw" width="160"> |
|
|
</a> |
|
|
<div>3.00 bpw</div> |
|
|
</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/3.04bpw.svg"> |
|
|
<img src="3.04bpw.svg" alt="3.04 bpw" width="160"> |
|
|
</a> |
|
|
<div>3.04 bpw</div> |
|
|
</td> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/3.5bpw.svg"> |
|
|
<img src="3.5bpw.svg" alt="3.50 bpw" width="160"> |
|
|
</a> |
|
|
<div>3.50 bpw</div> |
|
|
</td> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/4.0bpw.svg"> |
|
|
<img src="4.0bpw.svg" alt="4.00 bpw" width="160"> |
|
|
</a> |
|
|
<div>4.00 bpw</div> |
|
|
</td> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/4.03bpw.svg"> |
|
|
<img src="4.03bpw.svg" alt="4.00 bpw" width="160"> |
|
|
</a> |
|
|
<div>4.03 bpw</div> |
|
|
</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td align="center"> |
|
|
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/api.svg"> |
|
|
<img src="api.svg" alt="API" width="160"> |
|
|
</a> |
|
|
<div>API</div> |
|
|
</td> |
|
|
</tr> |
|
|
</table> |
