Article
Introducing AutoRound: Intel’s Advanced Quantization for LLMs and VLMs
•
40
wenhua cheng
wenhuach
AI & ML interests
Model Compression, CV
Recent Activity
posted
an
update
1 day ago
🚀 AutoRound(https://github.com/intel/auto-round) is now supported by SGLang!
After integrations with TorchAO, Transformers, and VLLM, AutoRound-quantized models are now officially compatible with SGLang — bringing faster and more flexible deployment to your LLM workflows.
💡 We’ve also enhanced the RTN mode (--iters 0), cutting quantization costs significantly for low-resource users.
⭐ Star our repo and stay tuned for more exciting updates!
reacted
to
their
post
with 🚀
1 day ago
AutoRound keeps evolving its LLM quantization algorithm! 🚀
After enhancing W2A16 quantization, we now offer a fast algorithm to generate mixed bits/data-type schemes (~2mins for 8B models), great for MXFP4 and W2A16.
Learn more: https://github.com/intel/auto-round/blob/main/docs/step_by_step.md#autoscheme
Organizations
published
an
article
6 months ago