[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71181":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":21,"hasPages":23,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":36,"readmeContent":37,"aiSummary":38,"trendingCount":16,"starSnapshotCount":16,"syncStatus":39,"lastSyncTime":40,"discoverSource":41},71181,"xtuner","InternLM\u002Fxtuner","InternLM","A Next-Generation Training Engine Built for Ultra-Large MoE Models","https:\u002F\u002Fxtuner.readthedocs.io\u002Fzh-cn\u002Flatest\u002F",null,"Python",5151,426,35,238,0,11,23,71.69,"Apache License 2.0",false,"main",true,[25,26,27,28,29,30,31,32,33,34,35],"agent","deepseek-v3","gpt-oss","intern-s1","internvl","kimi-k2","llm","multimodal","qwen3-moe","qwen3-vl","reinforcement-learning","2026-06-12 04:00:59","\u003Cdiv align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002FInternLM\u002Flmdeploy\u002Fassets\u002F36994684\u002F0cf8d00f-e86b-40ba-9b54-dc8f1bc6c8d8\" width=\"600\"\u002F>\n  \u003Cbr \u002F>\u003Cbr \u002F>\n\n[![GitHub Repo stars](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002FInternLM\u002Fxtuner?style=social)](https:\u002F\u002Fgithub.com\u002FInternLM\u002Fxtuner\u002Fstargazers)\n[![license](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flicense\u002FInternLM\u002Fxtuner.svg)](https:\u002F\u002Fgithub.com\u002FInternLM\u002Fxtuner\u002Fblob\u002Fmain\u002FLICENSE)\n[![PyPI](https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fv\u002Fxtuner)](https:\u002F\u002Fpypi.org\u002Fproject\u002Fxtuner\u002F)\n[![Downloads](https:\u002F\u002Fstatic.pepy.tech\u002Fbadge\u002Fxtuner)](https:\u002F\u002Fpypi.org\u002Fproject\u002Fxtuner\u002F)\n[![issue resolution](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues-closed-raw\u002FInternLM\u002Fxtuner)](https:\u002F\u002Fgithub.com\u002FInternLM\u002Fxtuner\u002Fissues)\n[![open issues](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues-raw\u002FInternLM\u002Fxtuner)](https:\u002F\u002Fgithub.com\u002FInternLM\u002Fxtuner\u002Fissues)\n\n👋 join us on [![Static Badge](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F-grey?style=social&logo=wechat&label=WeChat)](https:\u002F\u002Fcdn.vansin.top\u002Finternlm\u002Fxtuner.jpg)\n[![Static Badge](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F-grey?style=social&logo=twitter&label=Twitter)](https:\u002F\u002Ftwitter.com\u002Fintern_lm)\n[![Static Badge](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F-grey?style=social&logo=discord&label=Discord)](https:\u002F\u002Fdiscord.gg\u002Fxa29JuW87d)\n\n🔍 Explore our models on\n[![Static Badge](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F-gery?style=social&label=🤗%20Huggingface)](https:\u002F\u002Fhuggingface.co\u002Fxtuner)\n[![Static Badge](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F-gery?style=social&label=🤖%20ModelScope)](https:\u002F\u002Fwww.modelscope.cn\u002Forganization\u002Fxtuner)\n[![Static Badge](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F-gery?style=social&label=🧰%20OpenXLab)](https:\u002F\u002Fopenxlab.org.cn\u002Fusercenter\u002Fxtuner)\n[![Static Badge](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F-gery?style=social&label=🧠%20WiseModel)](https:\u002F\u002Fwww.wisemodel.cn\u002Forganization\u002Fxtuner)\n\nEnglish | [简体中文](README_zh-CN.md)\n\n\u003C\u002Fdiv>\n\n## 🚀 Speed Benchmark\n\n\u003Cdiv align=center>\n  \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Ffa42d587-068d-427b-b88c-25a164b3511c\" style=\"width:80%\">\n\u003C\u002Fdiv>\n\n## 🎉 News\n\n- **\\[2025\u002F09\\]** XTuner V1 Released! A Next-Generation Training Engine Built for Ultra-Large MoE Models\n\n## 📖 XTuner V1\n\nXTuner V1 is a next-generation LLM training engine specifically designed for ultra-large-scale MoE models. Unlike traditional 3D parallel training architectures, XTuner V1 is optimized for the mainstream MoE training scenarios prevalent in today's academic research.\n\n### Key Features\n**📊 Dropless Training**\n\t\n  - **Scalable without complexity:** Train 200B-scale MoE models without expert parallelism; 600B models require only intra-node expert parallelism\t\n  - **Optimized parallelism strategy:** Smaller expert parallelism dimension compared to traditional 3D approaches, enabling more efficient Dropless training\n\n**📝 Long Sequence Support**\n\t\n  - **Memory-efficient design:** Train 200B MoE models on 64k sequence lengths without sequence parallelism through advanced memory optimization techniques\t\n  - **Flexible scaling:** Full support for DeepSpeed Ulysses sequence parallelism with linearly scalable maximum sequence length\t\n  - **Robust performance:** Maintains stability despite expert load imbalance during long sequence training\n\n**⚡ Superior Efficiency**\n\n  - **Massive scale:** Supports MoE training up to 1T parameters\t\n  - **Breakthrough performance:** First to achieve FSDP training throughput that surpasses traditional 3D parallel schemes for MoE models above 200B scale\n  - **Hardware optimization:** Achieves training efficiency on Ascend A3 Supernode that exceeds NVIDIA H800\n\n\n\u003Cdiv align=center>\n  \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fc4fb2bb4-56bd-4f1c-8188-7f5370314cf8\" style=\"width:90%\">\n\u003C\u002Fdiv>\n\n\n## 🔥 Roadmap\n\nXTuner V1 is committed to continuously improving training efficiency for pre-training, instruction fine-tuning, and reinforcement learning of ultra-large MoE models, with special focus on Ascend NPU optimization.\n\n### 🚀 Training Engine\n\nOur vision is to establish XTuner V1 as a versatile training backend that seamlessly integrates with the broader open-source ecosystem.\n\n\n|   Model    |  GPU(FP8) | GPU(BF16)| NPU(BF16) |\n|------------|-----------|----------|-----------|\n| Intern S1  |    ✅     |    ✅    |    ✅     |\n| Intern VL  |    ✅     |    ✅    |    ✅     |\n| Qwen3 Dense|    ✅     |    ✅    |    ✅     |\n| Qwen3 MoE  |    ✅     |    ✅    |    ✅     |\n| GPT OSS    |    ✅     |    ✅    |    🚧     |\n| Deepseek V3|    ✅     |    ✅    |    🚧     |\n| KIMI K2    |    ✅     |    ✅    |    🚧     |\n\n\n### 🧠 Algorithm\n\nThe algorithm component is actively evolving. We welcome community contributions - with XTuner V1, scale your algorithms to unprecedented sizes!\n\n**Implemented**\n\n\n- ✅ **Multimodal Pre-training** - Full support for vision-language model training\n- ✅ **Multimodal Supervised Fine-tuning** - Optimized for instruction following\t\n- ✅ [GRPO](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2402.03300) - Group Relative Policy Optimization\n\n\n**Coming Soon**\n\n- 🔄 [MPO](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2411.10442) - Mixed Preference Optimization\n- 🔄 [DAPO](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2503.14476) - Dynamic Sampling Policy Optimization\n- 🔄 **Multi-turn Agentic RL** - Advanced agent training capabilities\n\n\n### ⚡ Inference Engine Integration\n\nSeamless deployment with leading inference frameworks:\n- [x] LMDeploy\n- [ ] vLLM\n- [ ] SGLang\n\n\n\n### Data Preparation\n\n- You can use [GraphGen](https:\u002F\u002Fgithub.com\u002Fopen-sciencelab\u002FGraphGen) to create synthetic data for fine-tuning.\n\n## 🤝 Contributing\n\nWe appreciate all contributions to XTuner. Please refer to [CONTRIBUTING.md](.github\u002FCONTRIBUTING.md) for the contributing guideline.\n\n## 🙏 Acknowledgement\n\nThe development of XTuner V1's training engine has been greatly inspired by and built upon the excellent work of the open-source community. We extend our sincere gratitude to the following pioneering projects:\n\n**Training Engine:**\n\n- [Torchtitan](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Ftorchtitan) - A PyTorch native platform for training generative AI models\n- [Deepspeed](https:\u002F\u002Fgithub.com\u002Fdeepspeedai\u002FDeepSpeed) - Microsoft's deep learning optimization library\t\n- [MindSpeed](https:\u002F\u002Fgitee.com\u002Fascend\u002FMindSpeed) - Ascend's high-performance training acceleration library\t\n- [Megatron](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FMegatron-LM) - NVIDIA's large-scale transformer training framework\n\n\n**Reinforcement Learning:**\n\nXTuner V1's reinforcement learning capabilities have been enhanced through insights and best practices from:\n\n- [veRL](https:\u002F\u002Fgithub.com\u002Fvolcengine\u002Fverl) - Volcano Engine Reinforcement Learning for LLMs\t\n- [SLIME](https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime) - THU's scalable RLHF implementation\t\n- [AReal](https:\u002F\u002Fgithub.com\u002FinclusionAI\u002FAReaL) - Ant Reasoning Reinforcement Learning for LLMs\n- [OpenRLHF](https:\u002F\u002Fgithub.com\u002FOpenRLHF\u002FOpenRLHF) - An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray\n\nWe are deeply grateful to all contributors and maintainers of these projects for advancing the field of large-scale model training.\n\n\n## 🖊️ Citation\n\n```bibtex\n@misc{2023xtuner,\n    title={XTuner: A Toolkit for Efficiently Fine-tuning LLM},\n    author={XTuner Contributors},\n    howpublished = {\\url{https:\u002F\u002Fgithub.com\u002FInternLM\u002Fxtuner}},\n    year={2023}\n}\n```\n\n## License\n\nThis project is released under the [Apache License 2.0](LICENSE). Please also adhere to the Licenses of models and datasets being used.\n","InternLM\u002Fxtuner 是一个专为超大规模混合专家模型（MoE）设计的下一代训练引擎。其核心功能包括无损训练和长序列支持，通过优化的并行策略实现200B规模的MoE模型在不使用专家并行的情况下进行训练，600B模型仅需节点内专家并行；同时，借助先进的内存优化技术，该引擎能够在64k序列长度上训练200B MoE模型而无需序列并行。此外，它还全面支持DeepSpeed Ulysses序列扩展。适用于需要高效处理大规模多模态数据及复杂自然语言任务的研究与开发场景。",2,"2026-06-11 03:36:28","high_star"]