[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92482":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":16,"stars30d":17,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":18,"rankGlobal":10,"rankLanguage":10,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":20,"hasPages":20,"topics":22,"createdAt":10,"pushedAt":10,"updatedAt":23,"readmeContent":24,"aiSummary":25,"trendingCount":16,"starSnapshotCount":16,"syncStatus":26,"lastSyncTime":27,"discoverSource":28},92482,"lingbot-vla-v2","Robbyant\u002Flingbot-vla-v2","Robbyant","From Foundation to Application","",null,"Python",501,33,112,1,0,241,58.59,"Apache License 2.0",false,"main",[],"2026-07-22 04:02:06","\u003Ch1 align=\"center\">LingBot-VLA 2.0: From Foundation to Application\u003C\u002Fh1>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2607.06403\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fstatic\u002Fv1?label=Paper&message=PDF&color=red&logo=arxiv\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Ftechnology.robbyant.com\u002Flingbot-vla-v2\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FProject-Website-blue\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Frobbyant\u002Flingbot-vla-v2\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fstatic\u002Fv1?label=%F0%9F%A4%97%20Model&message=HuggingFace&color=yellow\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fmodelscope.cn\u002Fcollections\u002FRobbyant\u002FLingBot-VLA-V2\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fstatic\u002Fv1?label=%F0%9F%A4%96%20Model&message=ModelScope&color=purple\">\u003C\u002Fa>\n  \u003Ca href=\"LICENSE\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-Apache--2.0-green\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n## Overview\n\n**LingBot-VLA 2.0** is a practical Vision-Language-Action foundation model designed to move from large-scale pre-training toward reliable real-world robot applications.\n\nCompared with LingBot-VLA 1.0, LingBot-VLA 2.0 improves three core capabilities:\n\n- **Generalization across tasks and embodiments**: a redesigned data pipeline curates around **60,000 hours** of pre-training data, including **50,000 hours** of robot trajectories across **20 robot configurations** and **10,000 hours** of egocentric human videos.\n- **Expanded action space**: the unified representation supports arms, end-effectors, grippers, dexterous hands, waist, head, and mobile-base signals instead of only standard dual-arm manipulation.\n- **Predictive dynamics modeling**: future prediction is used as a proxy task, with DINO-Video providing semantic temporal priors and LingBot-Depth providing geometric cues.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flingbot_vla2_framework.png\" width=\"86%\">\n\u003C\u002Fp>\n\n## News\n\n- **[2026-07-08]** LingBot-VLA 2.0 technical report and pre-trained weights are prepared.\n\n## Installation\n\nRequirements:\n\n- Miniconda or Anaconda\n- Python 3.12\n- PyTorch 2.8.0\n\nBefore running the setup script, make sure Conda is initialized in your shell and `conda activate` works.\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FRobbyant\u002Flingbot-vla-v2.git\ncd lingbot-vla-v2\n\nbash tools\u002Fcreate_train_env.sh\n```\n\nBy default, the script installs `flash-attn==2.8.3` from pip. If you already have a matching local wheel, pass it explicitly:\n\n```bash\nbash tools\u002Fcreate_train_env.sh \\\n  --flash-attn-wheel \u002Fpath\u002Fto\u002Fflash_attn-2.8.3+cu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl\n```\n\nYou can also choose the environment name or force a rebuild:\n\n```bash\nbash tools\u002Fcreate_train_env.sh \\\n  --env-name lingbotvla \\\n  --recreate\n```\n\n## Model Download\n\nWe release **LingBot-VLA 2.0** pre-trained weights as a native-depth model.\n\n| Model Name | Hugging Face | ModelScope | Description |\n| :--- | :---: | :---: | :---: |\n| LingBot-VLA 2.0 | [lingbot-vla-v2-6b](https:\u002F\u002Fhuggingface.co\u002Frobbyant\u002Flingbot-vla-v2-6b) | [lingbot-vla-v2-6b](https:\u002F\u002Fmodelscope.cn\u002Fmodels\u002FRobbyant\u002Flingbot-vla-v2-6b) | Native Depth |\n\nTo train LingBot-VLA 2.0 with this codebase, weights from [Qwen3-VL-4B-Instruct](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-VL-4B-Instruct), [MoGe-2-vitb-normal](https:\u002F\u002Fhuggingface.co\u002FRuicheng\u002Fmoge-2-vitb-normal), [LingBot-Depth](https:\u002F\u002Fhuggingface.co\u002Frobbyant\u002Flingbot-vla-v2-6b\u002Ftree\u002Fmain\u002Fdepth), and [DINO-VIDEO](https:\u002F\u002Fhuggingface.co\u002Frobbyant\u002Flingbot-vla-v2-6b\u002Ftree\u002Fmain\u002Fdino_video) teacher checkpoint\u002Fconfig are also required. See [Training_Config.md](configs\u002Fvla\u002FTraining_Config.md).\n\n```bash\npython3 scripts\u002Fdownload_hf_model.py --repo_id robbyant\u002Flingbot-vla-v2-6b --local_dir lingbot-vla\n```\n\n## Pre-Training Data\n\nLingBot-VLA 2.0 uses a large, heterogeneous pre-training corpus that covers single-arm, dual-arm, half-humanoid, humanoid, and egocentric sources.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flingbot_vla2_data_demo.png\" width=\"100%\">\n\u003C\u002Fp>\n\nThe raw pool is filtered into high-quality robotic and egocentric streams. The robotic side removes video-state misalignment, blurry or occluded videos, multi-view misalignment, abnormal velocity\u002Facceleration\u002Fjerk, and static-signal episodes. The egocentric side keeps manipulation-centric videos, reconstructs and standardizes hand trajectories, and filters unstable camera or hand-motion estimates.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flingbot_vla2_data_process.png\" width=\"70%\">\n\u003C\u002Fp>\n\n## Model Design\n\n### Unified Action Representation\n\nLingBot-VLA 2.0 maps heterogeneous embodiments into a 55-dimensional canonical state\u002Faction vector:\n\n- 14 dimensions for arm joint position\n- 14 dimensions for end-effector pose\n- 2 dimensions for gripper position\n- 12 dimensions for hand joint position\n- 4 dimensions for waist position\n- 2 dimensions for head position\n- 3 dimensions for mobility signal\n- 4 reserved dimensions\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flingbot_vla2_data_dimension.png\" width=\"100%\">\n\u003C\u002Fp>\n\n### MoE Action Expert\n\nTo improve cross-embodiment scaling, LingBot-VLA 2.0 uses sparse MoE layers inside the action expert. Fine-grained expert segmentation and shared expert isolation allow universal priors and specialized embodiment\u002Ftask patterns to coexist under the same active compute budget.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flingbot_vla2_loss_mse_comparison.png\" width=\"90%\">\n\u003C\u002Fp>\n\n### Dual-Query Distillation\n\nLingBot-VLA 2.0 appends current and future perceptual queries to the visual\u002Ftext tokens. These queries are distilled from LingBot-Depth and DINO-Video, encouraging causal inference to capture both current scene geometry and future scene evolution.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flingbot_vla2_vis_distillation.png\" width=\"100%\">\n\u003C\u002Fp>\n\n## Post-Training Example\n\n### Data Preparation\n\nPost-training requires three preparation steps. For a complete guide on customizing your own dataset, see the [Custom Data Guide](lingbotvla\u002Fdata\u002Fvla_data\u002FREADME.md).\n\n| Step | Description | Output |\n|------|-------------|--------|\n| 1. Prepare LeRobot Dataset | Prepare a LeRobot v2.1 or v3.0 dataset directory | LeRobot dataset directory |\n| 2. Prepare Robot Config | Define feature mapping from raw states\u002Factions\u002Fimages to the unified feature space | `configs\u002Frobot_configs\u002F\u003Cdata_name>.yaml` |\n| 3. Compute Norm Statistics | Calculate normalization statistics over your dataset | `assets\u002Fnorm_stats\u002F\u003Cname>.json` |\n\nBelow we use **RoboTwin 2.0** 50 tasks, trained with clean and randomized data together, as an example.\n\n- **Step 1 - RoboTwin Data**: Follow [RoboTwin2.0 Preparation](experiment\u002Frobotwin\u002FREADME.md) to download and prepare the dataset.\n- **Step 2 - Robot Config**: See [configs\u002Frobot_configs\u002Frobotwin.yaml](configs\u002Frobot_configs\u002Frobotwin.yaml) for the RoboTwin feature mapping.\n- **Step 3 - Normalization**: Pre-computed stats are provided at `assets\u002Fnorm_stats\u002Frobotwin.json`. To recompute for a custom task subset, see the [Custom Data Guide](lingbotvla\u002Fdata\u002Fvla_data\u002FREADME.md).\n\n### Training\n\nWe provide a post-training example of **LingBot-VLA 2.0** on RoboTwin 2.0 50 tasks with clean and randomized data:\n\n```bash\nbash train.sh tasks\u002Fvla\u002Ftrain_lingbotvla.py .\u002Fconfigs\u002Fvla\u002Frobotwin\u002Frobotwin.yaml \\\n  --data.train_path assets\u002Ftraining_data\u002Frobotwin.txt \\\n  --data.data_name multi \\\n  --train.output_dir output\u002F\n```\n\nThe post-training config uses sequence-wise auxiliary loss (`sequence_wise_mode: \"per_sequence\"`, `sequence_wise_loss_coeff: 1e-3`) together with z-loss (`router_z_loss_coeff: 1e-4`) for MoE routing. These terms can be adjusted or disabled depending on the downstream task. To use a loss-free routing setup, comment out the sequence-wise auxiliary loss and z-loss options, and set `bias_update_speed: 0.00025`.\nThe post-training config also enables the Muon optimizer. Muon can produce a better-converged loss, but it increases training time. To use the default AdamW optimizer instead, comment out `optimizer: muon`.\n\nFor real-world scenarios, see the native-depth training configuration [real_robot.yaml](configs\u002Fvla\u002Freal_robot\u002Freal_robot.yaml). For detailed explanations of batch size, gradient accumulation, checkpointing, depth\u002Fvideo distillation, MoE, and optimizer settings, see [Training_Config.md](configs\u002Fvla\u002FTraining_Config.md).\n\n## Evaluation and Deployment\n\n### Open-Loop Evaluation\n\n```bash\nexport QWEN3_PATH=Qwen\u002FQwen3-VL-4B-Instruct\npython scripts\u002Fopen_loop_eval.py \\\n  --model_path path_to_posttraining_ckpt \\\n  --robo_name robotwin \\\n  --data_path path_to_validation_data \\\n  --use_length 50\n```\n\n`--robo_name` is required for open-loop evaluation. It selects the robot config from `configs\u002Frobot_configs\u002F{robo_name}.yaml`, for example `--robo_name robotwin` uses `configs\u002Frobot_configs\u002Frobotwin.yaml`.\n\n\n\n### RoboTwin Deployment\n\nAfter making the RoboTwin simulation dependencies compatible with the model inference dependencies in a single environment, we provide a one-command evaluation script for all 50 RoboTwin 2.0 tasks:\n```bash\nQWEN3VL_PATH=\u002Fpath\u002Fto\u002FQwen3-VL-4B-Instruct\u002F \\\nEVAL_WORKDIR=\u002Fpath\u002Fto\u002FRobotwin_code\u002F \\\nbash experiment\u002Frobotwin\u002Fstart_robotwin_infer_and_eval.sh \\\n  --model_path \u002Fpath\u002Fto\u002Fyour\u002Fpost_training_checkpoint \\\n  --output_base \u002Fpath\u002Fto\u002Fyour\u002Feval_output \\\n  --num_per_gpu 2\n```\n`num_per_gpu` specifies how many tasks can be evaluated concurrently on each GPU. Tune it according to your available GPU memory and the communication load your machine can handle.\n\n### Real-Robot Deployment\n\n```bash\nexport QWEN3VL_PATH=path_to_Qwen3-VL-4B-Instruct\npython -m deploy.lingbot_vla_v2_policy \\\n  --model_path path_to_posttraining_ckpt \\\n  --use_compile \\\n  --use_length 25 \\\n  --port port\n```\n\nUsing `deploy.lingbot_vla_v2_policy`, one inference call on an NVIDIA GeForce RTX 4090D takes about **130 ms** with **10 denoising steps**.\n\n## Performance\n\nLingBot-VLA 2.0 is evaluated in a generalist setting on GM-100 bimanual manipulation and long-horizon mobile manipulation. Metrics are reported as progress score \u002F success rate where applicable.\n\n### GM-100 Bimanual Manipulation\n\n| Platform | GR00T N1.7 | π\u003Csub>0.5 | LingBot-VLA-1.0 | LingBot-VLA 2.0 |\n| :--- | ---: | ---: | ---: | ---: |\n| AgileX Cobot Magic | 36.3 \u002F 17.8 | 59.1 \u002F 32.2 | 58.2 \u002F 30.0 | **66.2 \u002F 34.4** |\n| Galaxea R1Pro | 16.4 \u002F 5.6 | 27.4 \u002F 8.9 | 32.7 \u002F **15.6** | **34.6 \u002F 15.6** |\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flingbot_vla2_gm100_ablation_barplot.png\" width=\"75%\">\n\u003C\u002Fp>\n\n### Long-Horizon Mobile Manipulation\n\n| Embodiment | Task | Setting | LingBot-VLA 2.0 | π\u003Csub>0.5 |\n| :--- | :--- | :--- | ---: | ---: |\n| Astribot S1 | Refrigerator sorting | In-domain | **77.1 \u002F 60.0** | 65.3 \u002F 46.7 |\n| Astribot S1 | Refrigerator sorting | Out-of-domain | **37.0 \u002F 13.3** | 30.3 \u002F 6.7 |\n| Cobot Magic-ARX X5 | Stove cleaning | In-domain | **84.3 \u002F 66.7** | 79.9 \u002F 60.0 |\n| Cobot Magic-ARX X5 | Stove cleaning | Out-of-domain | **67.5 \u002F 40.0** | 62.5 \u002F 33.3 |\n\n## Citation\n\nIf you find our work useful in your research, please cite:\n\n```bibtex\n@article{lingbotvla2,\n      title={From Foundation to Application: Improving VLA Models in Practice}, \n      author={Wei Wu and Fangjing Wang and Fan Lu and He Sun and Shi Liu and Yunnan Wang and Yibin Yan and Yong Wang and Shuailei Ma and Xinyang Wang and Yibin Liu and Shuai Yang and Tianxiang Zhou and Kejia Zhang and Lei Zhou and Cheng Su and Nan Xue and Bin Tan and Han Zhang and Youchao Zhang and Fei Liao and Xing Zhu and Yujun Shen and Kecheng Zheng},\n      journal={arXiv preprint arXiv:2607.06403},\n      year={2026}\n}\n```\n\n## License\n\nThis project is licensed under the [Apache-2.0 License](LICENSE).\n\n## Acknowledgement\n\nWe sincerely thank the developers of [VeOmni](https:\u002F\u002Farxiv.org\u002Fabs\u002F2508.02317) and [LeRobot](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Flerobot). This project benefits from their contributions to the open-source community.\n","LingBot-VLA 2.0 是一个面向真实机器人应用的视觉-语言-动作（VLA）基础模型。它通过融合50,000小时多构型机器人轨迹与10,000小时第一人称人类视频数据，支持跨任务、跨本体泛化；扩展了涵盖机械臂、灵巧手、移动底盘等多模态动作空间；并引入基于DINO-Video和LingBot-Depth的预测式动力学建模机制。适用于具身智能研究、服务机器人任务规划、端到端机器人控制等需要视觉理解、语言指令解析与连续动作生成协同的场景。",2,"2026-07-09 02:30:07","CREATED_QUERY"]