File size: 3,413 Bytes
a0879b1 9905486 a0879b1 9905486 a0879b1 8182b82 a0879b1 9905486 e309371 9905486 9c500e6 309881e dee6ef3 0a99308 309881e dee6ef3 0a99308 309881e 9a41e03 309881e bd09c56 dee6ef3 0a99308 bd09c56 dee6ef3 0a99308 bd09c56 309881e bd09c56 dee6ef3 0a99308 bd09c56 dee6ef3 0a99308 bd09c56 dee6ef3 0a99308 bd09c56 dee6ef3 0a99308 bd09c56 415929c bd09c56 dee6ef3 0a99308 bd09c56 309881e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
---
license: mit
base_model: MiniMaxAI/MiniMax-M2
base_model_relation: quantized
quantized_by: turboderp
tags:
- exl3
---
EXL3 quants of [MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2)
⚠️ Requires ExLlamaV3 v0.0.12 (or v0.0.11 `dev` branch)
Base bitrates:
[2.00 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/2.0bpw)
[3.00 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/3.0bpw)
[4.00 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/4.0bpw)
Optimized:
[2.04 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/2.04bpw)
[2.27 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/2.27bpw)
[3.04 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/3.04bpw)
[3.50 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/3.5bpw)
[4.03 bits per weight](https://huggingface.co/turboderp/MiniMax-M2-exl3/tree/4.03bpw)
. | KL-div | ppl | HumanEval@1
---------|--------|-------|-------------
2.00 bpw | 0.400 | 10.92 | 80.5%
2.04 bpw | 0.297 | 10.23 | 87.1%
2.27 bpw | 0.252 | 9.78 | 88.4%
3.00 bpw | 0.141 | 8.99 | 87.8%
3.04 bpw | 0.117 | 8.73 | 87.2%
3.50 bpw | 0.094 | 8.78 | 88.4%
4.00 bpw | 0.087 | 8.58 | 89.6%
4.03 bpw | 0.077 | 8.61 | 87.8%
original | - | 8.51 | 87.2%¹
¹ Unconfirmed
<table>
<tr>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/2.0bpw.svg">
<img src="2.0bpw.svg" alt="2.00 bpw" width="160">
</a>
<div>2.00 bpw</div>
</td>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/2.04bpw.svg">
<img src="2.04bpw.svg" alt="2.04 bpw" width="160">
</a>
<div>2.04 bpw</div>
</td>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/2.27bpw.svg">
<img src="2.27bpw.svg" alt="2.27 bpw" width="160">
</a>
<div>2.27 bpw</div>
</td>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/3.0bpw.svg">
<img src="3.0bpw.svg" alt="3.00 bpw" width="160">
</a>
<div>3.00 bpw</div>
</td>
</tr>
<tr>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/3.04bpw.svg">
<img src="3.04bpw.svg" alt="3.04 bpw" width="160">
</a>
<div>3.04 bpw</div>
</td>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/3.5bpw.svg">
<img src="3.5bpw.svg" alt="3.50 bpw" width="160">
</a>
<div>3.50 bpw</div>
</td>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/4.0bpw.svg">
<img src="4.0bpw.svg" alt="4.00 bpw" width="160">
</a>
<div>4.00 bpw</div>
</td>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/4.03bpw.svg">
<img src="4.03bpw.svg" alt="4.00 bpw" width="160">
</a>
<div>4.03 bpw</div>
</td>
</tr>
<tr>
<td align="center">
<a href="https://huggingface.co/turboderp/MiniMax-M2-exl3/blob/main/api.svg">
<img src="api.svg" alt="API" width="160">
</a>
<div>API</div>
</td>
</tr>
</table> |