[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92680":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":16,"stars30d":17,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":18,"rankGlobal":10,"rankLanguage":10,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":20,"hasPages":20,"topics":22,"createdAt":10,"pushedAt":10,"updatedAt":26,"readmeContent":27,"aiSummary":28,"trendingCount":16,"starSnapshotCount":16,"syncStatus":14,"lastSyncTime":29,"discoverSource":30},92680,"lingbot-video","Robbyant\u002Flingbot-video","Robbyant","Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence","https:\u002F\u002Ftechnology.robbyant.com\u002Flingbot-video",null,"Python",778,30,2,4,0,263,58.47,"Apache License 2.0",false,"master",[23,5,24,25],"image-to-video","video-generation","world-models","2026-07-22 04:02:06","# LingBot-Video\n\n**🌐 [Project Page](https:\u002F\u002Ftechnology.robbyant.com\u002Flingbot-video)** | **🤗 [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Frobbyant\u002Flingbot-video)** | **🤖 [ModelScope](https:\u002F\u002Fwww.modelscope.cn\u002Fcollections\u002FRobbyant\u002FLingBot-Video)** | **📄 [Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.07675)** | **⚖️ [License](LICENSE)**| **💬 [WeChat 微信 Group](assets\u002FWeChatGroup.JPG)** \n\n**📘 English Usage**: [English Documentation](docs\u002Fen\u002Findex.md) \\\n**📕 中文使用文档**: [中文文档](docs\u002Fzh\u002Findex.md)\n\nWe are excited to introduce **LingBot-Video**, the first open-source large-scale MoE (Mixture-of-Experts) video generation model dedicated to embodied intelligence. As a top-tier video model, LingBot-Video is designed to bridge the gap between video synthesis and physical world understanding.\n\n## 🔥 Key Highlights\n\n* **🚀 Efficient MoE Architecture**: Scaled from scratch; balanced between capacity and cost with **~3x** faster inference.\n* **📦 Data Engine**: Trained on massive web videos integrated with **70,000+ hours** of embodied data.\n* **⚖️ Multi Reward System**: Rewarded for **high aesthetics**, **physical rationality**, and **task completion**.\n\n## 🎬 Video Demos\n\n\u003Cdiv align=\"center\">\n  \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fe960aa83-6e5c-42ee-ba33-9fa25749ac03\" width=\"100%\" poster=\"\"> \u003C\u002Fvideo>\n\u003C\u002Fdiv>\n\n## 🔥 Latest News\n\n- July 9, 2026: 🎉 We release the technical report, code, models, rewriters for LingBot-Video.\n\n## 📦 Model Download\n\n| Model Name | Components | Tasks | Download |\n| --- | --- | --- | --- |\n| ⚡ LingBot-Video-Dense | Dense (1.3B) | T2I, T2V, TI2V | 🤗 [Huggingface](https:\u002F\u002Fhuggingface.co\u002Frobbyant\u002Flingbot-video-dense-1.3b)   🤖 [ModelScope](https:\u002F\u002Fwww.modelscope.cn\u002Fmodels\u002FRobbyant\u002Flingbot-video-dense-1.3b) |\n| 💪 LingBot-Video-MoE | MoE (30B-A3B) + Refiner | T2I, T2V, TI2V, Refinement | 🤗 [Huggingface](https:\u002F\u002Fhuggingface.co\u002Frobbyant\u002Flingbot-video-moe-30b-a3b)   🤖 [ModelScope](https:\u002F\u002Fwww.modelscope.cn\u002Fmodels\u002FRobbyant\u002Flingbot-video-moe-30b-a3b) |\n| 📝 LingBot-Video-Rewriter-Base | Qwen3.6-27B official | Prompt rewriter (Expand) | 🤗 [Huggingface](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3.6-27B)   🤖 [ModelScope](https:\u002F\u002Fwww.modelscope.cn\u002Fmodels\u002FQwen\u002FQwen3.6-27B) |\n| 📝 LingBot-Video-Rewriter-Adapter | Qwen3.6-27B LoRA | Prompt rewriter (Json) | 🤗 [Huggingface](https:\u002F\u002Fhuggingface.co\u002Frobbyant\u002Flingbot-video-rewriter-lora)   🤖 [ModelScope](https:\u002F\u002Fwww.modelscope.cn\u002Fmodels\u002FRobbyant\u002Flingbot-video-rewriter-lora) |\n\n## 🚀 Quick Start\n\n### 🛠️ Installation\n\nThe root `requirements.txt` includes the recommended PyTorch build for LingBot-Video inference.\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FRobbyant\u002Flingbot-video\ncd lingbot-video\n\npython -m venv .venv\nsource .venv\u002Fbin\u002Factivate\npython -m pip install -U pip\n\n# Base requirements cover direct DiT inference and rewriter --backend transformers.\npip install -r requirements.txt\npip install -e .\n```\n\n> **💡 Rewriter deployment**: the bundled rewriter uses the single-process\n> `transformers` backend. For higher throughput, deploy the VLM yourself and call\n> it through an OpenAI-compatible API. Preserve the two-stage semantics: step 1\n> must use the base VLM without the rewriter LoRA, while step 2 must use the same\n> base VLM with the rewriter LoRA enabled. This can be implemented with two\n> endpoints, or with one server that can select the adapter per request. See\n> [vLLM](https:\u002F\u002Fdocs.vllm.ai) \u002F [SGLang](https:\u002F\u002Fdocs.sglang.ai) official docs.\n> Details in [Prompt Preparation](docs\u002Fen\u002Fprompt_preparation.md#prompt-rewriter).\n\nInstall the optional SGLang dependencies only when using SGLang Diffusion or the\nfused \u002F FP8 MoE runtime:\n\n```bash\npython -m pip install --no-deps -r requirements-sglang.txt\n```\n\nRecommended runtime versions:\n\n| Package | Version |\n| --- | --- |\n| `Python` | `>=3.10` |\n| `torch` | `2.12.0.dev20260220+cu130` (recommended) |\n| `torchvision` | `0.26.0.dev20260220+cu130` (recommended) |\n| `transformers` | `5.8.1` |\n| `diffusers` | `0.39.0` |\n| `peft` | `0.19.1` |\n| `json_repair` | `>=0.30` |\n| `decord` | `>=0.6.0` |\n| `safetensors` | `>=0.4.5` |\n\n### 🎬 Inference\n\n#### 🧭 Recommended Inference Workflow\n\nLingBot-Video DiT inference is designed to consume structured JSON captions,\nnot casual natural-language prompts. The recommended public workflow is:\n\n1. Rewrite the user's plain prompt with\n   [Prompt Rewriter](docs\u002Fen\u002Fprompt_preparation.md#prompt-rewriter).\n   For TI2V, pass the same first frame to the rewriter.\n2. Run [Auto Negative](docs\u002Fen\u002Fprompt_preparation.md#auto-negative-prompt) by\n   default to prune the negative prompt for this specific caption.\n3. Run the unified inference runner with `--prompt_json` and select direct\n   diffusers or SGLang Diffusion through `--backend`.\n\nBackend choices:\n\n- `diffusers`: direct diffusers reference path.\n- `sglang`: SGLang Diffusion path. If the optional SGLang package is not\n  installed, it automatically falls back to direct diffusers and prints a\n  warning. Install `requirements-sglang.txt` to enable the SGLang runtime.\n\nFor multi-GPU inference, add `--enable_fsdp_inference` to shard the base DiT and\nrefiner DiT on GPU. This reduces GPU memory pressure after loading, but each\nrank still constructs the transformer on host memory before FSDP sharding; make\nsure the machine has enough system RAM for large MoE checkpoints.\n\n```bash\n# Model root (released Dense or MoE package) and rewriter weights.\nexport MODEL_DIR=\"\u003Cpath_to_lingbot-video-model>\"\nexport REWRITER_BASE_MODEL=\"\u003Cpath_to_rewriter_base_vlm>\"\nexport REWRITER_ADAPTER=\"\u003Cpath_to_rewriter_lora>\"\n\npython rewriter\u002Finference.py --backend transformers --mode t2v \\\n  --prompt \"\u003Cplain_user_prompt>\" --duration 5 --output prompt.json\n\n# Recommended Auto Negative block. If skipped, remove --negative_prompt_json from\n# the DiT inference command.\npython rewriter\u002Fauto_negative.py --backend transformers --mode t2v \\\n  --caption prompt.json --output negative.json\n\nexport BACKEND=diffusers  # or: sglang\n\npython scripts\u002Finference.py \\\n  --backend \"$BACKEND\" \\\n  --model_dir \"$MODEL_DIR\" \\\n  --run_refiner \\\n  --mode t2v \\\n  --prompt_json prompt.json \\\n  --negative_prompt_json negative.json \\\n  --output \"\u003Coutput_dir>\u002Fbase.mp4\" \\\n  --refiner_output \"\u003Coutput_dir>\u002Frefined.mp4\" \\\n  --height 480 \\\n  --width 832 \\\n  --fps 24 \\\n  --steps 40 \\\n  --refiner_steps 8 \\\n  --guidance_scale 3 \\\n  --refiner_guidance_scale 3 \\\n  --shift 3 \\\n  --refiner_shift 3 \\\n  --transformer_dtype bf16 \\\n  --text_encoder_dtype bf16 \\\n  --vae_dtype fp32 \\\n  --refiner_vae_dtype fp32 \\\n  --reuse_condition_features\n```\n\nReady-to-run scripts are provided for single-GPU and multi-GPU inference. Set\nyour environment and model path first:\n\n```bash\nsource .venv\u002Fbin\u002Factivate\nexport PYTHON_BIN=python\nexport DENSE_MODEL_DIR=\"\u003Cpath_to_lingbot-video-dense>\"\nexport MOE_MODEL_DIR=\"\u003Cpath_to_lingbot-video-moe>\"\n```\n\nSingle-GPU scripts use direct diffusers and batched CFG by default. They run\nbase generation only.\n\n```bash\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fsingle-gpu\u002Frun_dense_t2i.sh\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fsingle-gpu\u002Frun_dense_t2v.sh\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fsingle-gpu\u002Frun_dense_ti2v.sh\n\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fsingle-gpu\u002Frun_moe_t2i.sh\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fsingle-gpu\u002Frun_moe_t2v.sh\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fsingle-gpu\u002Frun_moe_ti2v.sh\n```\n\nMulti-GPU no-refiner scripts use the same inference arguments as the single-GPU\nscripts, plus CP8 and FSDP:\n\n```bash\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus-no-refiner\u002Frun_dense_t2i_fsdp_cp8.sh\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus-no-refiner\u002Frun_dense_t2v_fsdp_cp8.sh\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus-no-refiner\u002Frun_dense_ti2v_fsdp_cp8.sh\n\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus-no-refiner\u002Frun_moe_t2i_fsdp_cp8.sh\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus-no-refiner\u002Frun_moe_t2v_fsdp_cp8.sh\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus-no-refiner\u002Frun_moe_ti2v_fsdp_cp8.sh\n```\n\nMulti-GPU refiner scripts use CP8 + FSDP + batched CFG by default. They also\ndefault to direct diffusers; set `BACKEND=sglang` externally when you want to\nexercise SGLang Diffusion. MoE multi-GPU T2V\u002FTI2V scripts additionally run the\nrefiner.\n\n```bash\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus\u002Frun_dense_t2i_fsdp_cp8.sh\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus\u002Frun_dense_t2v_fsdp_cp8.sh\nMODEL_DIR=\"$DENSE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus\u002Frun_dense_ti2v_fsdp_cp8.sh\n\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus\u002Frun_moe_t2i_fsdp_cp8.sh\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus\u002Frun_moe_t2v_refiner_fsdp_cp8.sh\nMODEL_DIR=\"$MOE_MODEL_DIR\" .\u002Fscripts\u002Fmulti-gpus\u002Frun_moe_ti2v_refiner_fsdp_cp8.sh\n```\n\nAll scripts accept the same environment overrides, such as `PROMPT_JSON`,\n`IMAGE`, `OUT_DIR`, `HEIGHT`, `WIDTH`, `STEPS`, `GUIDANCE_SCALE`, `SHIFT`,\n`SEED`, `FPS`, `BACKEND`, and `PYTHON_BIN`. Refiner scripts also accept\n`REFINER_HEIGHT`, `REFINER_WIDTH`, `REFINER_STEPS`,\n`REFINER_GUIDANCE_SCALE`, `REFINER_SHIFT`, `REFINER_T_THRESH`, and\n`REFINER_SIGMA_TAIL_STEPS`. MoE scripts default to grouped expert execution\n(`LINGBOT_MOE_EXPERT_BACKEND=grouped_mm`).\n\nSee [English Docs](docs\u002Fen\u002Findex.md) or [中文文档](docs\u002Fzh\u002Findex.md) for the\ndetailed prompt rewrite, auto-negative, TI2V, base-only\u002Frefiner, distributed\nSGLang, and speed-first FP8 workflows.\n\n## 📊 Benchmarks\n\n### 🏛️ Public Benchmark\n\nAs of July 9th, 2026, LingBot-Video ranks top in [RBench Leaderboard](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FDAGroup-PKU\u002FRBench-Leaderboard).\n\n| Models | Open-source | Avg. | Manip. | Spatial | Multi-entity | Long-hor. | Reasoning | Single arm | Dual arm | Quadruped | Humanoid |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| **LingBot-Video (Ours)** | ✅ | **0.620** | **0.578** | \u003Cu>0.643\u003C\u002Fu> | 0.444 | **0.634** | \u003Cu>0.505\u003C\u002Fu> | 0.636 | 0.639 | **0.758** | 0.689 |\n| Cosmos3 Super | ✅ | 0.581 | 0.487 | 0.642 | 0.444 | \u003Cu>0.591\u003C\u002Fu> | 0.395 | 0.615 | 0.623 | \u003Cu>0.739\u003C\u002Fu> | \u003Cu>0.691\u003C\u002Fu> |\n| LongCat-Video | ✅ | 0.437 | 0.372 | 0.310 | 0.220 | 0.384 | 0.186 | 0.586 | 0.576 | 0.681 | 0.621 |\n| Wan 2.2 A14B | ✅ | 0.507 | 0.381 | 0.454 | 0.373 | 0.501 | 0.330 | 0.608 | 0.582 | 0.690 | 0.648 |\n| HunyuanVideo 1.5 | ✅ | 0.460 | 0.442 | 0.316 | 0.312 | 0.438 | 0.364 | 0.513 | 0.526 | 0.634 | 0.595 |\n| Wan 2.6 | ❌ | \u003Cu>0.607\u003C\u002Fu> | 0.546 | **0.656** | \u003Cu>0.479\u003C\u002Fu> | 0.514 | **0.531** | **0.666** | **0.681** | 0.723 | 0.667 |\n| Seedance 1.5 pro | ❌ | 0.584 | \u003Cu>0.577\u003C\u002Fu> | 0.495 | **0.484** | 0.570 | 0.470 | \u003Cu>0.648\u003C\u002Fu> | \u003Cu>0.641\u003C\u002Fu> | 0.680 | **0.692** |\n| Veo 3 | ❌ | 0.563 | 0.521 | 0.508 | 0.430 | 0.530 | 0.504 | 0.634 | 0.610 | 0.689 | 0.637 |\n\n\n*Note: **Bold** indicates the best performance, and \u003Cu>underline\u003C\u002Fu> indicates the second best.*\n\n### 🔬 Internal Benchmark\n\nOur internal evaluation comprehensively assesses both **Text-to-Video (T2V)** and **Text-to-Image-to-Video (TI2V)** capabilities for open-source models across two dimensions: **Quality Scores** (Motion, Prompt Following, Visual Consistency, Aesthetics) and **Domain Scores** (Human Interaction, Physics, Robot, Egocentric, Navigation).\n\n\u003C!-- 2x2 Image Grid Layout -->\n\u003Ctable border=\"0\" style=\"border-collapse: collapse; border: none; width: 100%;\">\n  \u003Ctr style=\"border: none;\">\n    \u003Ctd width=\"50%\" style=\"border: none; text-align: center; padding: 5px;\">\n      \u003Cimg src=\"assets\u002Fbenchmarks\u002Finternal\u002Ft2v_quality_notitle.png\" width=\"100%\" alt=\"T2V Quality Score\">\u003Cbr>\n      \u003Cb>(a) T2V Quality Score\u003C\u002Fb>\n    \u003C\u002Ftd>\n    \u003Ctd width=\"50%\" style=\"border: none; text-align: center; padding: 5px;\">\n      \u003Cimg src=\"assets\u002Fbenchmarks\u002Finternal\u002Ft2v_domain_notitle.png\" width=\"100%\" alt=\"T2V Domain Score\">\u003Cbr>\n      \u003Cb>(b) T2V Domain Score\u003C\u002Fb>\n    \u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr style=\"border: none;\">\n    \u003Ctd width=\"50%\" style=\"border: none; text-align: center; padding: 5px;\">\n      \u003Cimg src=\"assets\u002Fbenchmarks\u002Finternal\u002Fti2v_quality_notitle.png\" width=\"100%\" alt=\"TI2V Quality Score\">\u003Cbr>\n      \u003Cb>(c) TI2V Quality Score\u003C\u002Fb>\n    \u003C\u002Ftd>\n    \u003Ctd width=\"50%\" style=\"border: none; text-align: center; padding: 5px;\">\n      \u003Cimg src=\"assets\u002Fbenchmarks\u002Finternal\u002Fti2v_domain_notitle.png\" width=\"100%\" alt=\"TI2V Domain Score\">\u003Cbr>\n      \u003Cb>(d) TI2V Domain Score\u003C\u002Fb>\n    \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n## ⚖️ License\nThis project is licensed under the Apache 2.0 License. Please refer to the [LICENSE file](LICENSE) for the full text, including details on rights and restrictions.\n\n## 📚 Citation\nIf you find this work useful for your research, please cite our paper:\n\n```bibtex\n@article{lingbot-video,\n  title = {Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence},\n  author = {Shuailei Ma and Jiaqi Liao and Xinyang Wang and Jingjing Wang and Chaoran Feng and Zijing Hu and Chong Bao and Zichen Xi and Yuqi Gan and Weisen Wang and Yanhong Zeng and Qin Zhao and Zifan Shi and Wei Wu and Hao Ouyang and Qiuyu Wang and Shangzhan Zhang and Jiahao Shao and Yipengjing Sun and Liangxiao Hu and Lunke Pan and Nan Xue and Kecheng Zheng and Yinghao Xu and Xing Zhu and Yujun Shen and Ka Leong Cheng},\n  journal={arXiv preprint arXiv:2607.07675},\n  year = {2026}\n}\n```\n","LingBot-Video 是一个面向具身智能的开源视频生成模型，基于大规模混合专家（MoE）架构实现高效视频预训练。核心功能包括文本\u002F图像到视频生成（T2V\u002FTI2V）、物理合理性建模、美学与任务完成多目标奖励优化，并支持Prompt重写增强输入质量。技术特点涵盖30B参数MoE主干、7万+小时具身数据训练、约3倍加速推理及端到端可微调设计。适用于机器人仿真、虚拟环境构建、具身AI训练世界模型等需要高保真、物理可信视频合成的研究与开发场景。","2026-07-10 02:30:08","CREATED_QUERY"]