[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72218":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":22,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":35,"readmeContent":36,"aiSummary":37,"trendingCount":16,"starSnapshotCount":16,"syncStatus":38,"lastSyncTime":39,"discoverSource":40},72218,"nunchaku","nunchaku-ai\u002Fnunchaku","nunchaku-ai","[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models","https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002F",null,"Python",3869,256,41,4,0,8,30,24,29.23,"Apache License 2.0",false,"main",true,[26,27,28,29,30,31,32,33,34],"comfyui","diffusion-models","flux","genai","iclr","iclr2025","lora","mlsys","quantization","2026-06-12 02:03:00","\u003Cdiv align=\"center\" id=\"nunchaku_logo\">\n  \u003Cimg src=\"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fnunchaku-ai\u002Fcdn\u002Fresolve\u002Fmain\u002Flogo\u002Fv2\u002Fnunchaku-compact-transparent.png\" alt=\"logo\" width=\"220\">\u003C\u002Fimg>\n\u003C\u002Fdiv>\n\u003Ch3 align=\"center\">\n\u003Ca href=\"http:\u002F\u002Farxiv.org\u002Fabs\u002F2411.05007\">\u003Cb>Paper\u003C\u002Fb>\u003C\u002Fa> | \u003Ca href=\"https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002F\">\u003Cb>Docs\u003C\u002Fb>\u003C\u002Fa> | \u003Ca href=\"https:\u002F\u002Fhanlab.mit.edu\u002Fprojects\u002Fsvdquant\">\u003Cb>Website\u003C\u002Fb>\u003C\u002Fa> | \u003Ca href=\"https:\u002F\u002Fhanlab.mit.edu\u002Fblog\u002Fsvdquant\">\u003Cb>Blog\u003C\u002Fb>\u003C\u002Fa> | \u003Ca href=\"https:\u002F\u002Fdemo.nunchaku.tech\u002F\">\u003Cb>Demo\u003C\u002Fb>\u003C\u002Fa> | \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fnunchaku-ai\">\u003Cb>Hugging Face\u003C\u002Fb>\u003C\u002Fa> | \u003Ca href=\"https:\u002F\u002Fmodelscope.cn\u002Forganization\u002Fnunchaku-tech\">\u003Cb>ModelScope\u003C\u002Fb>\u003C\u002Fa> | \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002FComfyUI-nunchaku\">\u003Cb>ComfyUI\u003C\u002Fb>\u003C\u002Fa>\n\u003C\u002Fh3>\n\u003Cdiv align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Ftrendshift.io\u002Frepositories\u002F13942\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Ftrendshift.io\u002Fapi\u002Fbadge\u002Frepositories\u002F13942\" alt=\"nunchaku-ai\u002Fnunchaku | Trendshift\" style=\"width: 120px; height: 26px;\" width=\"120\" height=\"26\"\u002F>\u003C\u002Fa>\n  \u003Ca href=https:\u002F\u002Fdiscord.gg\u002FWk6PnwX9Sm target=\"_blank\">\u003Cimg src=https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fdynamic\u002Fjson?url=https%3A%2F%2Fdiscord.com%2Fapi%2Finvites%2FWk6PnwX9Sm%3Fwith_counts%3Dtrue&query=%24.approximate_member_count&logo=discord&logoColor=white&label=Discord&color=green&suffix=%20total height=22px>\u003C\u002Fa>\n  \u003Ca href=https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fnunchaku-ai\u002Fcdn\u002Fresolve\u002Fmain\u002Fnunchaku\u002Fassets\u002Fwechat.jpg target=\"_blank\">\u003Cimg src=https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FWeChat-07C160?logo=wechat&logoColor=white height=22px>\u003C\u002Fa>\n  \u003Ca href=https:\u002F\u002Fdeepwiki.com\u002Fnunchaku-ai\u002Fnunchaku target=\"_blank\">\u003Cimg src=https:\u002F\u002Fdeepwiki.com\u002Fbadge.svg height=22px>\u003C\u002Fa>\n\u003C\u002Fdiv>\n\n**Nunchaku** is a high-performance inference engine optimized for 4-bit neural networks, as introduced in our paper [SVDQuant](http:\u002F\u002Farxiv.org\u002Fabs\u002F2411.05007). For the underlying quantization library, check out [DeepCompressor](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002Fdeepcompressor).\n\nJoin our user groups on [**Discord**](https:\u002F\u002Fdiscord.gg\u002FWk6PnwX9Sm) and [**WeChat**](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fnunchaku-ai\u002Fcdn\u002Fresolve\u002Fmain\u002Fnunchaku\u002Fassets\u002Fwechat.jpg) to engage in discussions with the community! More details can be found [here](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002Fnunchaku\u002Fissues\u002F149). If you have any questions, run into issues, or are interested in contributing, don’t hesitate to reach out!\n\n## News\n\n- **[2026-01-12]** 🚀 **v1.2.0 Released!** Enjoy a **20–30% Z-Image performance boost**, seamless **LoRA support with native ComfyUI nodes**, **INT4 support for 20-series GPUs**, and **compatibility with ComfyUI 0.7**!\n- **[2025-12-20]** 🔥 Release **4-bit [Tongyi-MAI\u002FZ-Image-Turbo](https:\u002F\u002Fhuggingface.co\u002FTongyi-MAI\u002FZ-Image-Turbo)**! Download on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fnunchaku-ai\u002Fnunchaku-z-image-turbo) or [ModelScope](https:\u002F\u002Fmodelscope.cn\u002Fmodels\u002Fnunchaku-tech\u002Fnunchaku-z-image-turbo), and try it with our [example script](examples\u002Fv1\u002Fz-image-turbo.py).\n- **[2025-09-25]** 🔥 Release **4-bit [4\u002F8-step lightning Qwen-Image-Edit-2509](https:\u002F\u002Fhuggingface.co\u002Flightx2v\u002FQwen-Image-Lightning)**! Download on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fnunchaku-ai\u002Fnunchaku-qwen-image-edit-2509) or [ModelScope](https:\u002F\u002Fmodelscope.cn\u002Fmodels\u002Fnunchaku-tech\u002Fnunchaku-qwen-image-edit-2509), and try it with our [example script](examples\u002Fv1\u002Fqwen-image-edit-2509-lightning.py).\n- **[2025-09-24]** 🔥 Released **4-bit [Qwen-Image-Edit-2509](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen-Image-Edit-2509)**! Models are available on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fnunchaku-ai\u002Fnunchaku-qwen-image-edit-2509). Try them out with our [example script](examples\u002Fv1\u002Fqwen-image-edit-2509.py). Lightning models will follow up!\n\n\u003Cdetails>\n\u003Csummary>More\u003C\u002Fsummary>\n\n- **[2025-09-09]** 🔥 Released [**4-bit Qwen-Image-Edit**](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen-Image-Edit) together with the [4\u002F8-step Lightning](https:\u002F\u002Fhuggingface.co\u002Flightx2v\u002FQwen-Image-Lightning) variants! Models are available on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fnunchaku-ai\u002Fnunchaku-qwen-image-edit). Try them out with our [example script](examples\u002Fv1\u002Fqwen-image-edit.py).\n- **[2025-09-04]** 🚀 Official release of **Nunchaku v1.0.0**! Qwen-Image now supports **asynchronous offloading**, reducing VRAM usage to as little as **3 GiB** with no performance loss. Check out the [tutorial](https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002Fusage\u002Fqwenimage.html) to get started.\n- **[2025-08-27]** 🔥 Release **4-bit [4\u002F8-step lightning Qwen-Image](https:\u002F\u002Fhuggingface.co\u002Flightx2v\u002FQwen-Image-Lightning)**! Download on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fnunchaku-ai\u002Fnunchaku-qwen-image) or [ModelScope](https:\u002F\u002Fmodelscope.cn\u002Fmodels\u002Fnunchaku-tech\u002Fnunchaku-qwen-image), and try it with our [example script](examples\u002Fv1\u002Fqwen-image-lightning.py).\n- **[2025-08-15]** 🔥 Our **4-bit Qwen-Image** models are now live on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fnunchaku-ai\u002Fnunchaku-qwen-image)! Get started with our [example script](examples\u002Fv1\u002Fqwen-image.py). *ComfyUI, LoRA, and CPU offloading support are coming soon!*\n- **[2025-08-15]** 🚀 The **Python backend** is now available! Explore our Pythonic FLUX models [here](nunchaku\u002Fmodels\u002Ftransformers\u002Ftransformer_flux_v2.py) and see the modular **4-bit linear layer** [here](nunchaku\u002Fmodels\u002Flinear.py).\n- **[2025-07-31]** 🚀 **[FLUX.1-Krea-dev](https:\u002F\u002Fwww.krea.ai\u002Fblog\u002Fflux-krea-open-source-release) is now supported!** Check out our new [example script](.\u002Fexamples\u002Fflux.1-krea-dev.py) to get started.\n- **[2025-07-13]** 🚀 The official [**Nunchaku documentation**](https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002F) is now live! Explore comprehensive guides and resources to help you get started.\n- **[2025-06-29]** 🔥 Support **FLUX.1-Kontext**! Try out our [example script](.\u002Fexamples\u002Fflux.1-kontext-dev.py) to see it in action! Our demo is available at this [link](https:\u002F\u002Fsvdquant.mit.edu\u002Fkontext\u002F)!\n- **[2025-06-01]** 🚀 **Release v0.3.0!** This update adds support for multiple-batch inference, [**ControlNet-Union-Pro 2.0**](https:\u002F\u002Fhuggingface.co\u002FShakker-Labs\u002FFLUX.1-dev-ControlNet-Union-Pro-2.0), initial integration of [**PuLID**](https:\u002F\u002Fgithub.com\u002FToTheBeginning\u002FPuLID), and introduces [**Double FB Cache**](examples\u002Fflux.1-dev-double_cache.py). You can now load Nunchaku FLUX models as a single file, and our upgraded [**4-bit T5 encoder**](https:\u002F\u002Fhuggingface.co\u002Fnunchaku-ai\u002Fnunchaku-t5) now matches **FP8 T5** in quality!\n- **[2025-04-16]** 🎥 Released tutorial videos in both [**English**](https:\u002F\u002Fyoutu.be\u002FYHAVe-oM7U8?si=cM9zaby_aEHiFXk0) and [**Chinese**](https:\u002F\u002Fwww.bilibili.com\u002Fvideo\u002FBV1BTocYjEk5\u002F?share_source=copy_web&vd_source=8926212fef622f25cc95380515ac74ee) to assist installation and usage.\n- **[2025-04-09]** 📢 Published the [April roadmap](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002Fnunchaku\u002Fissues\u002F266) and an [FAQ](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002Fnunchaku\u002Fdiscussions\u002F262) to help the community get started and stay up to date with Nunchaku’s development.\n- **[2025-04-05]** 🚀 **Nunchaku v0.2.0 released!** This release brings [**multi-LoRA**](examples\u002Fflux.1-dev-multiple-lora.py) and [**ControlNet**](examples\u002Fflux.1-dev-controlnet-union-pro.py) support with even faster performance powered by [**FP16 attention**](#fp16-attention) and [**First-Block Cache**](#first-block-cache). We've also added compatibility for [**20-series GPUs**](examples\u002Fflux.1-dev-turing.py) — Nunchaku is now more accessible than ever!\n- **[2025-03-07]** 🚀 **Nunchaku v0.1.4 Released!** We've supported [4-bit text encoder and per-layer CPU offloading](#Low-Memory-Inference), reducing FLUX's minimum memory requirement to just **4 GiB** while maintaining a **2–3× speedup**. This update also fixes various issues related to resolution, LoRA, pin memory, and runtime stability. Check out the release notes for full details!\n- **[2025-02-20]** 🚀 **Support NVFP4 precision on NVIDIA RTX 5090!** NVFP4 delivers superior image quality compared to INT4, offering **~3× speedup** on the RTX 5090 over BF16. Learn more in our [blog](https:\u002F\u002Fhanlab.mit.edu\u002Fblog\u002Fsvdquant-nvfp4), checkout [`examples`](.\u002Fexamples) for usage and try [our demo](https:\u002F\u002Fsvdquant.mit.edu\u002Fflux1-schnell\u002F) online!\n- **[2025-02-18]** 🔥 [**Customized LoRA conversion**](#Customized-LoRA) and [**model quantization**](#Customized-Model-Quantization) instructions are now available! **[ComfyUI](.\u002Fcomfyui)** workflows now support **customized LoRA**, along with **FLUX.1-Tools**!\n- **[2025-02-11]** 🎉 **[SVDQuant](http:\u002F\u002Farxiv.org\u002Fabs\u002F2411.05007) has been selected as a ICLR 2025 Spotlight! FLUX.1-tools Gradio demos are now available!** Check [here](#gradio-demos) for the usage details! Our new [depth-to-image demo](https:\u002F\u002Fsvdquant.mit.edu\u002Fflux1-depth-dev\u002F) is also online—try it out!\n- **[2025-02-04]** **🚀 4-bit [FLUX.1-tools](https:\u002F\u002Fblackforestlabs.ai\u002Fflux-1-tools\u002F) is here!** Enjoy a **2-3× speedup** over the original models. Check out the [examples](.\u002Fexamples) for usage. **ComfyUI integration is coming soon!**\n- **[2025-01-23]** 🚀 **4-bit [SANA](https:\u002F\u002Fnvlabs.github.io\u002FSana\u002F) support is here!** Experience a 2-3× speedup compared to the 16-bit model. Check out the [usage example](examples\u002Fsana1.6b_pag.py) and the [deployment guide](app\u002Fsana\u002Ft2i) for more details. Explore our live demo at [svdquant.mit.edu](https:\u002F\u002Fsvdquant.mit.edu)!\n- **[2025-01-22]** 🎉 [**SVDQuant**](http:\u002F\u002Farxiv.org\u002Fabs\u002F2411.05007) has been accepted to **ICLR 2025**!\n- **[2024-12-08]** Support [ComfyUI](https:\u002F\u002Fgithub.com\u002Fcomfyanonymous\u002FComfyUI). Please check [ComfyUI-nunchaku](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002FComfyUI-nunchaku) for the usage.\n- **[2024-11-07]** 🔥 Our latest **W4A4** Diffusion model quantization work [**SVDQuant**](https:\u002F\u002Fhanlab.mit.edu\u002Fprojects\u002Fsvdquant) is publicly released! Check [**DeepCompressor**](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002Fdeepcompressor) for the quantization library.\n\n\u003C\u002Fdetails>\n\n## Overview\n\n![teaser](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fnunchaku-ai\u002Fcdn\u002Fresolve\u002Fmain\u002Fnunchaku\u002Fassets\u002Fteaser.jpg)\n**Nunchaku** is a high-performance inference engine for low-bit neural networks. It implements **SVDQuant**, a post-training quantization technique for 4-bit weights and activations that well maintains visual fidelity. On 12B FLUX.1-dev, it achieves 3.6× memory reduction compared to the BF16 model. By eliminating CPU offloading, it offers 8.7× speedup over the 16-bit model when on a 16GB laptop 4090 GPU, 3× faster than the NF4 W4A16 baseline. On PixArt-∑, it demonstrates significantly superior visual quality over other W4A4 or even W4A8 baselines. \"E2E\" means the end-to-end latency including the text encoder and VAE decoder.\n\n**SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models**\u003Cbr>\n[Muyang Li](https:\u002F\u002Flmxyy.me)\\*, [Yujun Lin](https:\u002F\u002Fyujunlin.com)\\*, [Zhekai Zhang](https:\u002F\u002Fhanlab.mit.edu\u002Fteam\u002Fzhekai-zhang)\\*, [Tianle Cai](https:\u002F\u002Fwww.tianle.website\u002F#\u002F), [Xiuyu Li](https:\u002F\u002Fxiuyuli.com), [Junxian Guo](https:\u002F\u002Fgithub.com\u002FJerryGJX), [Enze Xie](https:\u002F\u002Fxieenze.github.io), [Chenlin Meng](https:\u002F\u002Fcs.stanford.edu\u002F~chenlin\u002F), [Jun-Yan Zhu](https:\u002F\u002Fwww.cs.cmu.edu\u002F~junyanz\u002F), and [Song Han](https:\u002F\u002Fhanlab.mit.edu\u002Fsonghan) \u003Cbr>\n*MIT, NVIDIA, CMU, Princeton, UC Berkeley, SJTU, and Pika Labs* \u003Cbr>\n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Ffdd4ab68-6489-4c65-8768-259bd866e8f8\n\n## Method\n\n#### Quantization Method -- SVDQuant\n\n![intuition](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fnunchaku-ai\u002Fcdn\u002Fresolve\u002Fmain\u002Fnunchaku\u002Fassets\u002Fintuition.gif)Overview of SVDQuant. Stage1: Originally, both the activation $\\boldsymbol{X}$ and weights $\\boldsymbol{W}$ contain outliers, making 4-bit quantization challenging. Stage 2: We migrate the outliers from activations to weights, resulting in the updated activation $\\hat{\\boldsymbol{X}}$ and weights $\\hat{\\boldsymbol{W}}$. While $\\hat{\\boldsymbol{X}}$ becomes easier to quantize, $\\hat{\\boldsymbol{W}}$ now becomes more difficult. Stage 3: SVDQuant further decomposes $\\hat{\\boldsymbol{W}}$ into a low-rank component $\\boldsymbol{L}_1\\boldsymbol{L}_2$ and a residual $\\hat{\\boldsymbol{W}}-\\boldsymbol{L}_1\\boldsymbol{L}_2$ with SVD. Thus, the quantization difficulty is alleviated by the low-rank branch, which runs at 16-bit precision.\n\n#### Nunchaku Engine Design\n\n![engine](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fnunchaku-ai\u002Fcdn\u002Fresolve\u002Fmain\u002Fnunchaku\u002Fassets\u002Fengine.jpg) (a) Naïvely running low-rank branch with rank 32 will introduce 57% latency overhead due to extra read of 16-bit inputs in *Down Projection* and extra write of 16-bit outputs in *Up Projection*. Nunchaku optimizes this overhead with kernel fusion. (b) *Down Projection* and *Quantize* kernels use the same input, while *Up Projection* and *4-Bit Compute* kernels share the same output. To reduce data movement overhead, we fuse the first two and the latter two kernels together.\n\n## Performance\n\n![efficiency](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fnunchaku-ai\u002Fcdn\u002Fresolve\u002Fmain\u002Fnunchaku\u002Fassets\u002Fefficiency.jpg)SVDQuant reduces the 12B FLUX.1 model size by 3.6× and cuts the 16-bit model's memory usage by 3.5×. With Nunchaku, our INT4 model runs 3.0× faster than the NF4 W4A16 baseline on both desktop and laptop NVIDIA RTX 4090 GPUs. Notably, on the laptop 4090, it achieves a total 10.1× speedup by eliminating CPU offloading. Our NVFP4 model is also 3.1× faster than both BF16 and NF4 on the RTX 5090 GPU.\n\n## Getting Started\n\n- [Installation Guide](https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002Finstallation\u002Finstallation.html)\n- [Usage Tutorial](https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002Fusage\u002Fbasic_usage.html)\n- [ComfyUI Plugin: ComfyUI-nunchaku](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002FComfyUI-nunchaku)\n- [Custom Model Quantization: DeepCompressor](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002Fdeepcompressor)\n- [Gradio Demo Apps](https:\u002F\u002Fgithub.com\u002Fnunchaku-ai\u002Fnunchaku\u002Ftree\u002Fmain\u002Fapp)\n- [Reproduce SVDQuant Paper Results](app\u002Fflux.1\u002Ft2i)\n- [API Reference](https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002Fpython_api\u002Fnunchaku.html)\n- [Contribution Guide](https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002Fdeveloper\u002Fcontribution_guide.html)\n- [Frequently Asked Questions](https:\u002F\u002Fnunchaku.tech\u002Fdocs\u002Fnunchaku\u002Ffaq\u002Ffaq.html)\n\n## Contact Us\n\nFor enterprises interested in adopting SVDQuant or Nunchaku, including technical consulting, sponsorship opportunities, or partnership inquiries, please contact us at muyangli@nunchaku.tech.\n\n## Related Projects\n\n- [Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models](https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.02048), NeurIPS 2022 & T-PAMI 2023\n- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.10438), ICML 2023\n- [Q-Diffusion: Quantizing Diffusion Models](https:\u002F\u002Farxiv.org\u002Fabs\u002F2302.04304), ICCV 2023\n- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.00978), MLSys 2024\n- [DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models](https:\u002F\u002Farxiv.org\u002Fabs\u002F2402.19481), CVPR 2024\n- [QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving](https:\u002F\u002Farxiv.org\u002Fabs\u002F2405.04532), MLSys 2025\n- [SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers](https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.10629), ICLR 2025\n- [Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity](https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.01776), ICML 2025\n- [Radial Attention: $O(n \\log n)$ Sparse Attention with Energy Decay for Long Video Generation](https:\u002F\u002Fgithub.com\u002Fmit-han-lab\u002Fradial-attention), NeurIPS 2025\n- [Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation](https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.18875), NeurIPS 2025\n\n## Citation\n\nIf you find `nunchaku` useful or relevant to your research, please cite our paper:\n\n```bibtex\n@inproceedings{\n  li2024svdquant,\n  title={SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models},\n  author={Li*, Muyang and Lin*, Yujun and Zhang*, Zhekai and Cai, Tianle and Li, Xiuyu and Guo, Junxian and Xie, Enze and Meng, Chenlin and Zhu, Jun-Yan and Han, Song},\n  booktitle={The Thirteenth International Conference on Learning Representations},\n  year={2025}\n}\n```\n\n## Acknowledgments\n\nWe thank MIT-IBM Watson AI Lab, MIT and Amazon Science Hub, MIT AI Hardware Program, National Science Foundation, Packard Foundation, Dell, LG, Hyundai, and Samsung for supporting this research. We thank NVIDIA for donating the DGX server. We thank [First Intelligence](https:\u002F\u002Fwww.first-intelligence.com\u002F) and [Yotta Labs](https:\u002F\u002Fwww.yottalabs.ai\u002F) for generously sponsoring our computing resources.\n\nWe use [img2img-turbo](https:\u002F\u002Fgithub.com\u002FGaParmar\u002Fimg2img-turbo) to train the sketch-to-image LoRA. Our text-to-image and image-to-image UI is built upon [playground-v.25](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fplaygroundai\u002Fplayground-v2.5\u002Fblob\u002Fmain\u002Fapp.py) and [img2img-turbo](https:\u002F\u002Fgithub.com\u002FGaParmar\u002Fimg2img-turbo\u002Fblob\u002Fmain\u002Fgradio_sketch2image.py), respectively. Our safety checker is borrowed from [hart](https:\u002F\u002Fgithub.com\u002Fmit-han-lab\u002Fhart).\n\nNunchaku is also inspired by many open-source libraries, including (but not limited to) [TensorRT-LLM](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FTensorRT-LLM), [vLLM](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm), [QServe](https:\u002F\u002Fgithub.com\u002Fmit-han-lab\u002Fqserve), [AWQ](https:\u002F\u002Fgithub.com\u002Fmit-han-lab\u002Fllm-awq), [FlashAttention-2](https:\u002F\u002Fgithub.com\u002FDao-AILab\u002Fflash-attention), and [Atom](https:\u002F\u002Fgithub.com\u002Fefeslab\u002FAtom).\n\n## Star History\n\n[![Star History Chart](https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=nunchaku-ai\u002Fnunchaku&type=Date)](https:\u002F\u002Fwww.star-history.com\u002F#nunchaku-ai\u002Fnunchaku&Date)\n","Nunchaku 是一个针对4位神经网络优化的高性能推理引擎，基于论文[SVDQuant](http:\u002F\u002Farxiv.org\u002Fabs\u002F2411.05007)提出。该项目通过低秩分量吸收异常值的方法实现了高效的4位量化模型，支持LoRA和ComfyUI，并且在最新版本中为20系列GPU提供了INT4支持。其核心功能包括显著提升Z-Image性能、无缝集成LoRA以及与ComfyUI 0.7兼容。Nunchaku适用于需要高效运行深度学习模型的应用场景，特别是在资源受限环境下追求快速推理速度的情况。",2,"2026-06-11 03:40:54","high_star"]