[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71037":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":23,"hasPages":23,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":31,"readmeContent":32,"aiSummary":33,"trendingCount":16,"starSnapshotCount":16,"syncStatus":34,"lastSyncTime":35,"discoverSource":36},71037,"bitsandbytes","bitsandbytes-foundation\u002Fbitsandbytes","bitsandbytes-foundation","Accessible large language models via k-bit quantization for PyTorch.","https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fbitsandbytes\u002Fmain\u002Fen\u002Findex",null,"Python",8264,867,52,32,0,5,20,73,15,39.82,"MIT License",false,"main",[26,27,28,29,30],"llm","machine-learning","pytorch","qlora","quantization","2026-06-12 02:02:46","\u003Cp align=\"center\">\u003Cimg src=\"https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F175231607?s=200&v=4\" alt=\"\">\u003C\u002Fp>\n\u003Ch1 align=\"center\">bitsandbytes\u003C\u002Fh1>\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fbitsandbytes-foundation\u002Fbitsandbytes\u002Fmain\u002FLICENSE\">\u003Cimg alt=\"License\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flicense\u002Fbitsandbytes-foundation\u002Fbitsandbytes.svg?color=blue\">\u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fpepy.tech\u002Fproject\u002Fbitsandbytes\">\u003Cimg alt=\"Downloads\" src=\"https:\u002F\u002Fstatic.pepy.tech\u002Fbadge\u002Fbitsandbytes\u002Fmonth\">\u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fbitsandbytes-foundation\u002Fbitsandbytes\u002Factions\u002Fworkflows\u002Ftests-nightly.yml\">\u003Cimg alt=\"Nightly Unit Tests\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Factions\u002Fworkflow\u002Fstatus\u002Fbitsandbytes-foundation\u002Fbitsandbytes\u002Ftests-nightly.yml?logo=github&label=Nightly%20Tests\">\u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fbitsandbytes-foundation\u002Fbitsandbytes\u002Freleases\">\u003Cimg alt=\"GitHub Release\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fv\u002Frelease\u002Fbitsandbytes-foundation\u002Fbitsandbytes\">\u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fbitsandbytes\u002F\">\u003Cimg alt=\"PyPI - Python Version\" src=\"https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fpyversions\u002Fbitsandbytes\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n`bitsandbytes` enables accessible large language models via k-bit quantization for PyTorch. We provide three main features for dramatically reducing memory consumption for inference and training:\n\n* 8-bit optimizers uses block-wise quantization to maintain 32-bit performance at a small fraction of the memory cost.\n* LLM.int8() or 8-bit quantization enables large language model inference with only half the required memory and without any performance degradation. This method is based on vector-wise quantization to quantize most features to 8-bits and separately treating outliers with 16-bit matrix multiplication.\n* QLoRA or 4-bit quantization enables large language model training with several memory-saving techniques that don't compromise performance. This method quantizes a model to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allow training.\n\nThe library includes quantization primitives for 8-bit & 4-bit operations, through `bitsandbytes.nn.Linear8bitLt` and `bitsandbytes.nn.Linear4bit` and 8-bit optimizers through `bitsandbytes.optim` module.\n\n## System Requirements\nbitsandbytes has the following minimum requirements for all platforms:\n\n* Python 3.10+\n* [PyTorch](https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F) 2.4+\n  * _Note: While we aim to provide wide backwards compatibility, we recommend using the latest version of PyTorch for the best experience._\n\n#### Accelerator support:\n\n\u003Csmall>Note: this table reflects the status of the current development branch. For the latest stable release, see the\n[document in the 0.49.2 tag](https:\u002F\u002Fgithub.com\u002Fbitsandbytes-foundation\u002Fbitsandbytes\u002Fblob\u002F0.49.2\u002FREADME.md#accelerator-support).\n\u003C\u002Fsmall>\n\n##### Legend:\n🚧 = In Development,\n〰️ = Partially Supported,\n✅ = Supported,\n🐢 = Slow Implementation Supported,\n❌ = Not Supported\n\n\u003Ctable>\n  \u003Cthead>\n    \u003Ctr>\n      \u003Cth>Platform\u003C\u002Fth>\n      \u003Cth>Accelerator\u003C\u002Fth>\n      \u003Cth>Hardware Requirements\u003C\u002Fth>\n      \u003Cth>LLM.int8()\u003C\u002Fth>\n      \u003Cth>QLoRA 4-bit\u003C\u002Fth>\n      \u003Cth>8-bit Optimizers\u003C\u002Fth>\n    \u003C\u002Ftr>\n  \u003C\u002Fthead>\n  \u003Ctbody>\n    \u003Ctr>\n      \u003Ctd colspan=\"6\">🐧 \u003Cstrong>Linux, glibc >= 2.24\u003C\u002Fstrong>\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd align=\"right\">x86-64\u003C\u002Ftd>\n      \u003Ctd>◻️ CPU\u003C\u002Ftd>\n      \u003Ctd>Minimum: AVX2\u003Cbr>Optimized: AVX512F, AVX512BF16\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>🟩 NVIDIA GPU \u003Cbr>\u003Ccode>cuda\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>SM60+ minimum\u003Cbr>SM75+ recommended\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>🟥 AMD GPU \u003Cbr>\u003Ccode>cuda\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>\n        CDNA: gfx90a, gfx942, gfx950\u003Cbr>\n        RDNA: gfx1100, gfx1101, gfx1102, gfx1103, gfx1150, gfx1151, gfx1152, gfx1153, gfx1200, gfx1201\n      \u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>🟦 Intel GPU \u003Cbr>\u003Ccode>xpu\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>\n        Data Center GPU Max Series\u003Cbr>\n        Arc A-Series (Alchemist)\u003Cbr>\n        Arc B-Series (Battlemage)\n      \u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>🟪 Intel Gaudi \u003Cbr>\u003Ccode>hpu\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>Gaudi2, Gaudi3\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>〰️\u003C\u002Ftd>\n      \u003Ctd>❌\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd align=\"right\">aarch64\u003C\u002Ftd>\n      \u003Ctd>◻️ CPU\u003C\u002Ftd>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>❌\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>🟩 NVIDIA GPU \u003Cbr>\u003Ccode>cuda\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>SM75+\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd colspan=\"6\">🪟 \u003Cstrong>Windows 11 \u002F Windows Server 2022+\u003C\u002Fstrong>\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd align=\"right\">x86-64\u003C\u002Ftd>\n      \u003Ctd>◻️ CPU\u003C\u002Ftd>\n      \u003Ctd>AVX2\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>🟩 NVIDIA GPU \u003Cbr>\u003Ccode>cuda\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>SM60+ minimum\u003Cbr>SM75+ recommended\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>🟥 AMD GPU \u003Cbr>\u003Ccode>cuda\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>\n        RDNA: gfx1100, gfx1101, gfx1102,\u003Cbr>\n        gfx1150, gfx1151,\u003Cbr>\n        gfx1200, gfx1201\n      \u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>🟦 Intel GPU \u003Cbr>\u003Ccode>xpu\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>\n        Arc A-Series (Alchemist) \u003Cbr>\n        Arc B-Series (Battlemage)\n      \u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd colspan=\"6\">🍎 \u003Cstrong>macOS 14+\u003C\u002Fstrong>\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd align=\"right\">arm64\u003C\u002Ftd>\n      \u003Ctd>◻️ CPU\u003C\u002Ftd>\n      \u003Ctd>Apple M1+\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>❌\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003C\u002Ftd>\n      \u003Ctd>⬜ Metal \u003Cbr>\u003Ccode>mps\u003C\u002Fcode>\u003C\u002Ftd>\n      \u003Ctd>Apple M1+\u003C\u002Ftd>\n      \u003Ctd>🐢\u003C\u002Ftd>\n      \u003Ctd>🐢\u003C\u002Ftd>\n      \u003Ctd>❌\u003C\u002Ftd>\n  \u003C\u002Ftbody>\n\u003C\u002Ftable>\n\n## :book: Documentation\n* [Official Documentation](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fbitsandbytes\u002Fmain)\n* 🤗 [Transformers](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Ftransformers\u002Fquantization\u002Fbitsandbytes)\n* 🤗 [Diffusers](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fdiffusers\u002Fquantization\u002Fbitsandbytes)\n* 🤗 [PEFT](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fpeft\u002Fdeveloper_guides\u002Fquantization#quantize-a-model)\n\n## :heart: Sponsors\nThe continued maintenance and development of `bitsandbytes` is made possible thanks to the generous support of our sponsors. Their contributions help ensure that we can keep improving the project and delivering valuable updates to the community.\n\n\u003Ckbd>\u003Ca href=\"https:\u002F\u002Fhf.co\" target=\"_blank\">\u003Cimg width=\"100\" src=\"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fhuggingface\u002Fbrand-assets\u002Fresolve\u002Fmain\u002Fhf-logo.svg\" alt=\"Hugging Face\">\u003C\u002Fa>\u003C\u002Fkbd>\n \n\u003Ckbd>\u003Ca href=\"https:\u002F\u002Fintel.com\" target=\"_blank\">\u003Cimg width=\"100\" src=\"https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F17888862?s=100&v=4\" alt=\"Intel\">\u003C\u002Fa>\u003C\u002Fkbd>\n\n## License\n`bitsandbytes` is MIT licensed.\n\n## How to cite us\nIf you found this library useful, please consider citing our work:\n\n### QLoRA\n\n```bibtex\n@article{dettmers2023qlora,\n  title={Qlora: Efficient finetuning of quantized llms},\n  author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},\n  journal={arXiv preprint arXiv:2305.14314},\n  year={2023}\n}\n```\n\n### LLM.int8()\n\n```bibtex\n@article{dettmers2022llmint8,\n  title={LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale},\n  author={Dettmers, Tim and Lewis, Mike and Belkada, Younes and Zettlemoyer, Luke},\n  journal={arXiv preprint arXiv:2208.07339},\n  year={2022}\n}\n```\n\n### 8-bit Optimizers\n\n```bibtex\n@article{dettmers2022optimizers,\n  title={8-bit Optimizers via Block-wise Quantization},\n  author={Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke},\n  journal={9th International Conference on Learning Representations, ICLR},\n  year={2022}\n}\n```\n","bitsandbytes 是一个通过 k 位量化技术使大型语言模型在 PyTorch 上更易于访问的库。其核心功能包括8位优化器、8位量化（LLM.int8()）和4位量化(QLoRA)，这些功能显著减少了推理和训练时的内存消耗，同时保持了高性能。其中，8位优化器利用块级量化以极低的内存成本维持32位性能；8位量化则通过矢量级量化减少内存使用而不损失性能；QLoRA 则通过将模型量化至4位并引入少量可训练的LoRA权重来实现高效训练。此项目特别适用于需要运行或微调大型语言模型但受硬件资源限制的场景，如个人开发者、研究机构及小型企业等。",2,"2026-06-11 03:35:34","high_star"]