[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-93568":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":16,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":17,"compositeScore":18,"rankGlobal":9,"rankLanguage":9,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":9,"pushedAt":9,"updatedAt":24,"readmeContent":25,"aiSummary":9,"trendingCount":15,"starSnapshotCount":15,"syncStatus":26,"lastSyncTime":27,"discoverSource":28},93568,"MiniCPM-Robot","OpenBMB\u002FMiniCPM-Robot","OpenBMB","A Smarter and Faster On-Device AI Brain for Robots",null,"Python",206,15,138,1,0,36,108,85.04,"Apache License 2.0",false,"main",true,[],"2026-07-22 04:02:09","\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fminicpm_robot.png\" width=\"400\" alt=\"MiniCPM-Robot\" \u002F>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cstrong>A Smarter and Faster On-Device AI Brain for Robots\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cb>English\u003C\u002Fb> · \u003Ca href=\".\u002FREADME_zh.md\">中文\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cdiv align=\"center\">\n\n[![Github](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGithub-000000?style=for-the-badge&logo=github&logoColor=white)](https:\u002F\u002Fgithub.com\u002FOpenBMB\u002FMiniCPM-Robot) [![Hugging Face Collection](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FHuggingFace-fcd022?style=for-the-badge&logo=huggingface&logoColor=000)](https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Fopenbmb\u002Fminicpm-robot) [![ModelScope](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FModelScope-7B61FF?style=for-the-badge&logo=modelscope)](https:\u002F\u002Fmodelscope.cn\u002Fcollections\u002FOpenBMB\u002FMiniCPM-Robot) [![Cookbook](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%8D%B3_cookbook-ff69b4?style=for-the-badge)](#quick-start) [![Lark](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLark-00D6B9?style=for-the-badge&logo=data:image\u002Fpng;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAIIklEQVR4nO2ba2xUxxXHf3Pvvte7Xhu\u002F8WKDH6SEuJDSRKGQiEeAhrbhFaVpRBMoJaFSKlVVWj5Ujfr4EqSqaqOoRGkhTdQWiFq1jUjLI5AQ8gClJQQSwFCDY7CNwcZr73qf904\u002F2AvGb+\u002FOspHin+QP3nvvmTP\u002FmTlzztxdIaWUfI7Rsu1AtpkQINsOZJsJAbLtQLaZECDbDmQbSyoPxaTJ5XiYRIZTCKum4dNsuDUdIURG2khJgOZ4D8+2nqItEVPvUT8sCKbaXGwsrGaq3Y1AvQgi1Uzwza5Wnm9v4GAkSFi5Wzez2O7m2bKZ3ObIVW475Rhwr6eYX5bOZLnTR06GpmeS\u002FdEQL1xtIGYaym2nPAOSdBlxXmg7y687LxEgszHhpeJaVudXoClcCmnvAl7dyhNFNWzO91Op6Wq8GoaXOy9SH+lSalPJNujVrWwqquVXpV+gRk8pro6JA9Ee\u002FtjegKlwpinLA6xCY4m3lJ8WVlOn21SZHcSO7nY+CLajqopXmghpCB70+fl5yfSMLYc2abKzs5EuI67EnvJMUBeCRZ4SflFcQ51uVW0egP2hAPXRbiW2MpIKa0Lwjb6Z4M\u002FATDhnJtjd2azEVsZqAQuCRZ5SnimcRrWmPjC+FrpKWzz9FEwDSab+NAErfX6e8E1W0un+tBgJDnVfSdtOxqtBh6bz7YJpPO0tJk9hxhhAciTUkXYwHDQ3DRN0xbLk6BY2FdXSnIjwp56AMrtnYyE6ElGscTjdEKC5Lcily93Ejd4t0qJpeD02yos9VPu9lBa7B9kQUpo3bajb9ggiCVi3GJx2dQmHBM5Hu3my6RjvxKPpGTNB7zbJ+V+UBRfcNJxsJxCME40ZhOMGos9tKcCma9itOg67BX+JmxULqrj7iyVUTfHicloHC7Bll2DvcXhsoWTtovT8HIhEsqerhc2tZzhrJFKzEZE4DwexHgtjno+CMf5BKivI4alH7mDd6hlDnwcEwpI\u002F7BfoOqyaa+KwqVm7gt4c4aloDz++eoHwOFJaS1wijvXgOBjCbIxiptDxJJpmMm9OOUKMEARDUcm2fSY7D2kkFFahVqGxwufnQbdnzM\u002FYgybO17uwvXINsyGS0qgnKfI5+dF35lBT4UUIMfIuEAgLXtwr+c3foTOUcpuDyLfa+H5BLffZHKPea+800be3I\u002F8dgJiZdtuPfn06Dy2tQdN6Z\u002FWo8T6agFffhxd3C6IxdUvhDqePx\u002FL8I96nB0xsuzrg4zAy\u002Fb5TNdnDxjUz0bUb\u002FRjThmcY8Ncjks3bof5i+o7Qly4v805mfU4eziGu69dMHNvbMf7bcz2qp4PdqrNhVR15PvvNfozVgCHh8BnJlr8JzqlJw\u002FFarDzim0LFgDMEW5eJ\u002FdUOxJmwks4DVJd7WDLPjzYgGRt3ynP8guS7z8E\u002F3xOE0kzFBXCPp4j1uTdSZf2aie337Yj\u002F9CiZ9kkeX3E7\u002FtKcQZ+nlPMFo\u002FDcbsm2vYKeNOOCAL7qK+M+qwPZN\u002FKcjaRlcyDVfi9L5lUMeS3lpLezB15+S7Lxt\u002FDeJwIjjdGqtOewRpYyaXtH38iry0DtVp0NK2dSXDBUpFFQDJ1plvxsh+SlfaktCSklTc1BDmw7j3FG\u002FRuGqWU5LJlfMWjtJ1FS9rQHYeseyabn4chpQXyMWa5hSI6eaGPdT\u002Fax951GEmkkOMOxfuVMyocogpIorftONcMzf+5NnFqvCcwRlkU8brD7zQv8cMshTjV0qHTjOrdV+rj\u002FK1NGvEf5UU17EHa+CwdPSB6eD8vvEkzy3jyyTS1BfrfjI\u002F7yr3pCkdSKotGw23TWfm0GZUXDjz6ZECBJWzds3QNvfQxL7xQs+5JEFwneePdTtu46wfH6duLpRM5RmDW9kOULpjHaGUzm3mIAcQM+apScbIR\u002FvA9ecZnX9x0lHA5msllcLjdPfvPLlBWOXmtkVIAkJlDfAlKW4Ju6CHtXM8Gr9RixENJUuwQsdhePrribhXcVje1+pa2PghA6NlchNlchnkk19AQuEu5sJBJsxUykeUoEWOweZs++m++tqcRpH8MDt1qA\u002FmhWFzkFtbjyKklEu4kEmugJNJGIdGEaMeQ48mBNt+LwTKas6k5+8K18JheM3Y+sCZBE023YXJOwuQrwFNcRDbURDV0m2n2ZaOgKZmL4tFizOnF4SnHnVZI7aQqPL7Ywb8b42s+6AP0Rmo7DU4rDU4JREMNIhDFiQWLBVuLREFKaaJoFi82F7sjF4S5CszixWa0snwNrF4BjnO9lP1MC3ECgW+zoFjs4fDi95cPfKWD+DNiwlDGv+\u002F58RgUYGxYN7pkOmx+CvBzZV1uO00ZGPLsFaBosmwUbH4B8Dyl1nqEEKPAp8C7D5Dph1VxYf\u002F\u002F41\u002FxABhVD824Htz2z3\u002FpKh3y35OnVvW+u0u08Q70aMyW88SG8chBOX4Jb93uSkUW3W2Hu9N5gV10GmqIxGiRAkotXYNfbgtc+kATVnlAN58qwV8rz4eF74YE54HUpbnU4AQAShqS5Q2PbHjh6TnJF7TfUBrpy03+aBtXFsLCud737chi1skup1ZEESBKNCU42woEPYd9xybUe9Y70F6CqGJbMgoWzoaIwMx2\u002F3upYBEhimpLOkODwScHbn0BjG7QEIBJLL1DkOgWl+TCjHBbUwewqsFrlsOd4KhmXAP0JRQUt7fDpVTjdBKebJGdbek+ERgucQsCUAkFloaRuKtSWCfyFUJrfO\u002FVvJSkLoJ7sbL2f+1+MTAiQbQeyzYQA2XYg20wIkG0Hss3\u002FAd9m22xHQgjPAAAAAElFTkSuQmCC)](assets\u002Ffeishu_group.png)\n\n\u003C\u002Fdiv>\n\n**MiniCPM-Robot** is MiniCPM's embodied intelligence model family for real-world perception, decision-making, and action. The first models include:\n\n- **MiniCPM-RobotManip**: 🦾 A **1.5B** generalist VLA for robot manipulation (sim & real). One set of weights across tasks; beats larger models such as π₀.₅ (3B) and Qwen-VLA (5B+) on representative evals. Streaming inference preserves native memory capabilities and maintains the same response speed as before.\n\n- **MiniCPM-RobotTrack**: 🎯 The **first fully on-device** embodied target tracker (**0.9B**). Covers static, dynamic, and adversarial targets; **open-source SOTA on EVT-Bench**; runs **5+ FPS \u002F ~180 ms** on Unitree Go2 EDU with fully local, vision-only natural-language tracking.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"MiniCPM-RobotManip\u002Fassets\u002Fmanip_case_en.gif\" width=\"800\" alt=\"MiniCPM-RobotManip task demonstrations\" \u002F>\n\u003C\u002Fp>\n\n## 🎉 News\n\n* [2026.07.19] 🔥🔥🔥 We release and open-source MiniCPM-Robot, MiniCPM's first embodied intelligence model family for interaction with the physical world. Its first releases are [MiniCPM-RobotManip](https:\u002F\u002Fhuggingface.co\u002Fopenbmb\u002FMiniCPM-RobotManip) for generalist robot manipulation and [MiniCPM-RobotTrack](https:\u002F\u002Fhuggingface.co\u002Fopenbmb\u002FMiniCPM-RobotTrack) for embodied target tracking. Try it now!\n\n* [2026.07.19] 🚀🚀🚀 [PhyAI](https:\u002F\u002Fgithub.com\u002FMEmbodied\u002Fphyai) adds Day-0 support for MiniCPM-Robot, increasing inference throughput on NVIDIA H20 from 10 Hz to 37 Hz through CUDA Graph and custom Triton fused kernels.\n\n## Contents\n\n- [MiniCPM-RobotManip](#minicpm-robotmanip)\n  - [Benchmark Results](#benchmark-results)\n  - [Quick Start](#quick-start)\n  - [Inference](#inference)\n- [MiniCPM-RobotTrack](#minicpm-robottrack)\n  - [Benchmark Results](#evt-bench-results)\n  - [Quick Start](#quick-start-1)\n  - [Cookbook](#data-preparation)\n- [Model Zoo](#model-zoo)\n\n## MiniCPM-RobotManip\n\u003Cstrong>MiniCPM-RobotManip\u003C\u002Fstrong> is a 1.5B vision-language-action model for embodied manipulation with the following highlights:\n\u003Cul>\n  \u003Cli>\u003Cb>Generalist Manipulation:\u003C\u002Fb> A unified 1.5B generalist policy that covers all downstream tasks with one set of weights.\u003C\u002Fli>\n  \u003Cli>\u003Cb>Streaming Context:\u003C\u002Fb> Historical observations are incorporated into the model context through streaming inference. With 60 frames of history, traditional recomputation requires 125 TFLOPs per decision step, while streaming inference needs only 3.3 TFLOPs. The model supports up to 1 minute of visual context memory while keeping online cost comparable to traditional single-frame reactive inference.\u003C\u002Fli>\n  \u003Cli>\u003Cb>Efficient Inference:\u003C\u002Fb> Inherits MiniCPM-V 4.6's visual token compression, reducing each frame from 256 to 64 visual tokens for 4× compression. With H100, BF16, and single-frame input, model-forward latency per decision step is 120 ms, compared with 234 ms for π0.5. The measurement excludes task autoregressive decoding.\u003C\u002Fli>\n\u003C\u002Ful>\n\n### Benchmark Results\n\n\u003Ctable align=\"center\">\n  \u003Cthead>\n    \u003Ctr>\n      \u003Cth rowspan=\"2\">Method\u003C\u002Fth>\n      \u003Cth rowspan=\"2\">Eval Setting\u003C\u002Fth>\n      \u003Cth rowspan=\"2\">Open Weights\u003C\u002Fth>\n      \u003Cth rowspan=\"2\">Model Size\u003C\u002Fth>\n      \u003Cth rowspan=\"2\">LIBERO\u003C\u002Fth>\n      \u003Cth rowspan=\"2\">Calvin (ABC→D)\u003C\u002Fth>\n      \u003Cth colspan=\"2\">RoboTwin2 (clean+random)\u003C\u002Fth>\n      \u003Cth rowspan=\"2\">RMBench\u003C\u002Fth>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Cth>easy\u003C\u002Fth>\u003Cth>hard\u003C\u002Fth>\n    \u003C\u002Ftr>\n  \u003C\u002Fthead>\n  \u003Ctbody>\n    \u003Ctr>\n      \u003Ctd>π₀\u003C\u002Ftd>\u003Ctd>Specialist\u003C\u002Ftd>\u003Ctd>✅\u003C\u002Ftd>\u003Ctd>3B\u003C\u002Ftd>\u003Ctd>94.4\u003C\u002Ftd>\u003Ctd>3.9\u003C\u002Ftd>\u003Ctd>65.9\u003C\u002Ftd>\u003Ctd>58.4\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>π₀.₅\u003C\u002Ftd>\u003Ctd>Specialist\u003C\u002Ftd>\u003Ctd>✅\u003C\u002Ftd>\u003Ctd>3B\u003C\u002Ftd>\u003Ctd>96.9\u003C\u002Ftd>\u003Ctd>4.1\u003C\u002Ftd>\u003Ctd>82.7\u003C\u002Ftd>\u003Ctd>76.8\u003C\u002Ftd>\u003Ctd>10.4\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>Abot-M0\u003C\u002Ftd>\u003Ctd>Specialist\u003C\u002Ftd>\u003Ctd>✅\u003C\u002Ftd>\u003Ctd>4B+\u003C\u002Ftd>\u003Ctd>98.6\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\u003Ctd>86.1\u003C\u002Ftd>\u003Ctd>85.1\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>StarVLA-α\u003C\u002Ftd>\u003Ctd>Generalist\u003C\u002Ftd>\u003Ctd>✅\u003C\u002Ftd>\u003Ctd>4B+\u003C\u002Ftd>\u003Ctd>97.8\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\u003Ctd>88.7\u003C\u002Ftd>\u003Ctd>87.8\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>Qwen-VLA\u003C\u002Ftd>\u003Ctd>Generalist\u003C\u002Ftd>\u003Ctd>❌\u003C\u002Ftd>\u003Ctd>5B+\u003C\u002Ftd>\u003Ctd>97.9\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\u003Ctd>86.1\u003C\u002Ftd>\u003Ctd>87.2\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>LingBot-VA\u003C\u002Ftd>\u003Ctd>Specialist\u003C\u002Ftd>\u003Ctd>✅\u003C\u002Ftd>\u003Ctd>5B+\u003C\u002Ftd>\u003Ctd>98.5\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\u003Ctd>92.9\u003C\u002Ftd>\u003Ctd>91.6\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003Cb>MiniCPM-RobotManip\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>Generalist\u003C\u002Ftd>\u003Ctd>✅\u003C\u002Ftd>\u003Ctd>\u003Cb>1.5B\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>97.5\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>4.1\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>91.3\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>91.6\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>53.3\u003C\u002Fb>\u003C\u002Ftd>\n    \u003C\u002Ftr>\n  \u003C\u002Ftbody>\n\u003C\u002Ftable>\n\n### Quick Start\n\n\u003Cp>Install and initialize Conda first. The specification follows the tested Python 3.10 and PyTorch 2.6.0 (CUDA 12.4) setup.\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotManip\nconda env create -f environment.yml\nconda activate MiniCPM-RobotManip\u003C\u002Fcode>\u003C\u002Fpre>\n\n### Inference\n\n\u003Cp>Run single-sample inference with \u003Ccode>vla_infer.py\u003C\u002Fcode>. Provide at least one image and a language instruction; robot state defaults to zeros if omitted. The model returns an action chunk of shape \u003Ccode>(30, 80)\u003C\u002Fcode>.\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotManip\npython vla_infer.py \\\n    --image frame.jpg \\\n    --text \"Pick up the red block.\" \\\n    --checkpoint .\u002Fcheckpoint \\\n    --state-file state.npy \\\n    --embodiment-id 0 \\\n    --output action.npy\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>Use multiple \u003Ccode>--image\u003C\u002Fcode> flags for multi-view inputs. Without \u003Ccode>--output\u003C\u002Fcode>, the predicted action is printed as JSON.\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotManip\npython vla_infer.py \\\n    --image cam_front.jpg \\\n    --image cam_wrist.jpg \\\n    --text \"Pick up the red block.\" \\\n    --checkpoint .\u002Fcheckpoint\u003C\u002Fcode>\u003C\u002Fpre>\n\n## MiniCPM-RobotTrack\n\u003Cstrong>MiniCPM-RobotTrack\u003C\u002Fstrong> is a compact vision-language-action policy for embodied target tracking built on MiniCPM4-0.5B with following highlights:\n\n\u003Cul>\n  \u003Cli>\u003Cb>Quality-driven self-evolving data pipeline:\u003C\u002Fb> automated checks and manual review remove abnormal trajectories, incorrect actions, and invalid interactions, while continual model-environment interaction adds high-value training samples.\u003C\u002Fli>\n  \u003Cli>\u003Cb>DAgger for embodied tracking:\u003C\u002Fb> after learning from large-scale general scenarios, the model interacts with simulators and real robots to expose failures in long-tail cases such as target crossings, rapid turns, short occlusions, and multi-person intersections. Samples corrected by expert policies or rules are aggregated into the next training round to continuously improve tracking and generalization.\u003C\u002Fli>\n  \u003Cli>\u003Cb>End-to-end Go2 optimization:\u003C\u002Fb> joint optimization across visual capture, input encoding, inference, action generation, command transmission, and execution delivers a stable \u003Cb>5+ FPS\u003C\u002Fb> with approximately \u003Cb>180 ms\u003C\u002Fb> end-to-end latency on the Unitree Go2's native onboard compute.\u003C\u002Fli>\n  \u003Cli>\u003Cb>One-command Go2 Edu deployment and launch:\u003C\u002Fb> the \u003Ca href=\"MiniCPM-RobotTrack\u002Fdocs\u002FGO2_DEPLOYMENT.md\">local deployment workflow\u003C\u002Fa> covers environment setup, dependencies, model loading, camera input, text commands, robot control, and a one-command deployment-and-launch workflow, enabling fully local, vision-only natural-language tracking without rebuilding a separate perception, planning, and control stack.\u003C\u002Fli>\n\u003C\u002Ful>\n\n### Examples\n\n\u003Ctable align=\"center\">\n  \u003Ctr>\n    \u003Ctd align=\"center\" width=\"33%\">\n      \u003Cimg src=\"MiniCPM-RobotTrack\u002Fassets\u002Ftrack1_en.gif\" width=\"100%\" alt=\"Outdoor obstacle-aware target-tracking demo\" \u002F>\n    \u003C\u002Ftd>\n    \u003Ctd align=\"center\" width=\"33%\">\n      \u003Cimg src=\"MiniCPM-RobotTrack\u002Fassets\u002Ftrack2_en.gif\" width=\"100%\" alt=\"Elevator target-tracking demo\" \u002F>\n    \u003C\u002Ftd>\n    \u003Ctd align=\"center\" width=\"33%\">\n      \u003Cimg src=\"MiniCPM-RobotTrack\u002Fassets\u002Ftrack3_en.gif\" width=\"100%\" alt=\"Underground parking target-tracking demo\" \u002F>\n    \u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd align=\"center\">\u003Cb>Outdoor Obstacle-aware Tracking\u003C\u002Fb>\u003C\u002Ftd>\n    \u003Ctd align=\"center\">\u003Cb>Elevator Tracking\u003C\u002Fb>\u003C\u002Ftd>\n    \u003Ctd align=\"center\">\u003Cb>Underground Parking Tracking\u003C\u002Fb>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n### EVT-Bench Results\n\n\u003Cp>\n  Results are reported as \u003Cb>SR \u002F TR \u002F CR\u003C\u002Fb>: success rate and tracking rate are higher-is-better, while collision rate is lower-is-better. All values are percentages.\n\u003C\u002Fp>\n\n\u003Ctable align=\"center\">\n  \u003Cthead>\n    \u003Ctr>\n      \u003Cth rowspan=\"2\">Method\u003C\u002Fth>\n      \u003Cth rowspan=\"2\">Open Weights\u003C\u002Fth>\n      \u003Cth rowspan=\"2\">Model Size\u003C\u002Fth>\n      \u003Cth colspan=\"3\">STT\u003C\u002Fth>\n      \u003Cth colspan=\"3\">DT\u003C\u002Fth>\n      \u003Cth colspan=\"3\">AT\u003C\u002Fth>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Cth>SR ↑\u003C\u002Fth>\u003Cth>TR ↑\u003C\u002Fth>\u003Cth>CR ↓\u003C\u002Fth>\n      \u003Cth>SR ↑\u003C\u002Fth>\u003Cth>TR ↑\u003C\u002Fth>\u003Cth>CR ↓\u003C\u002Fth>\n      \u003Cth>SR ↑\u003C\u002Fth>\u003Cth>TR ↑\u003C\u002Fth>\u003Cth>CR ↓\u003C\u002Fth>\n    \u003C\u002Ftr>\n  \u003C\u002Fthead>\n  \u003Ctbody>\n    \u003Ctr>\n      \u003Ctd>TrackVLA\u003C\u002Ftd>\n      \u003Ctd>❌\u003C\u002Ftd>\n      \u003Ctd>7.8B+\u003C\u002Ftd>\n      \u003Ctd>85.1\u003C\u002Ftd>\u003Ctd>78.6\u003C\u002Ftd>\u003Ctd>1.7\u003C\u002Ftd>\n      \u003Ctd>57.6\u003C\u002Ftd>\u003Ctd>63.2\u003C\u002Ftd>\u003Ctd>5.8\u003C\u002Ftd>\n      \u003Ctd>50.2\u003C\u002Ftd>\u003Ctd>63.7\u003C\u002Ftd>\u003Ctd>17.1\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>TrackVLA++\u003C\u002Ftd>\n      \u003Ctd>❌\u003C\u002Ftd>\n      \u003Ctd>7.8B+\u003C\u002Ftd>\n      \u003Ctd>86.0\u003C\u002Ftd>\u003Ctd>81.0\u003C\u002Ftd>\u003Ctd>2.1\u003C\u002Ftd>\n      \u003Ctd>66.5\u003C\u002Ftd>\u003Ctd>68.8\u003C\u002Ftd>\u003Ctd>4.7\u003C\u002Ftd>\n      \u003Ctd>51.2\u003C\u002Ftd>\u003Ctd>63.4\u003C\u002Ftd>\u003Ctd>15.9\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>Qwen-RobotNav\u003C\u002Ftd>\n      \u003Ctd>❌\u003C\u002Ftd>\n      \u003Ctd>4.4B+\u003C\u002Ftd>\n      \u003Ctd>77.4\u003C\u002Ftd>\u003Ctd>90.0\u003C\u002Ftd>\u003Ctd>6.4\u003C\u002Ftd>\n      \u003Ctd>—\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\n      \u003Ctd>—\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\u003Ctd>—\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>OmTrackVLA\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>1.0B+\u003C\u002Ftd>\n      \u003Ctd>81.4\u003C\u002Ftd>\u003Ctd>82.8\u003C\u002Ftd>\u003Ctd>5.1\u003C\u002Ftd>\n      \u003Ctd>41.5\u003C\u002Ftd>\u003Ctd>58.8\u003C\u002Ftd>\u003Ctd>11.3\u003C\u002Ftd>\n      \u003Ctd>60.0\u003C\u002Ftd>\u003Ctd>73.9\u003C\u002Ftd>\u003Ctd>7.6\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Ctd>\u003Cb>MiniCPM-RobotTrack\u003C\u002Fb>\u003C\u002Ftd>\n      \u003Ctd>✅\u003C\u002Ftd>\n      \u003Ctd>\u003Cb>0.9B\u003C\u002Fb>\u003C\u002Ftd>\n      \u003Ctd>\u003Cb>84.1\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>89.8\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>3.0\u003C\u002Fb>\u003C\u002Ftd>\n      \u003Ctd>\u003Cb>53.2\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>73.4\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>13.6\u003C\u002Fb>\u003C\u002Ftd>\n      \u003Ctd>\u003Cb>58.0\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>80.4\u003C\u002Fb>\u003C\u002Ftd>\u003Ctd>\u003Cb>9.0\u003C\u002Fb>\u003C\u002Ftd>\n    \u003C\u002Ftr>\n  \u003C\u002Ftbody>\n\u003C\u002Ftable>\n\n\u003Cp>\n  Among the open-source checkpoints shown above, MiniCPM-RobotTrack obtains the best STT SR\u002FTR\u002FCR and the best DT SR\u002FTR. It also reaches \u003Cb>80.35 TR\u003C\u002Fb> on AT with a 0.5B backbone.\n\u003C\u002Fp>\n\n### Quick Start\n\n#### 1. Create the environment\n\n\u003Cp>Install and initialize Conda first. The specification follows the tested Python 3.9, Habitat-Sim 0.3.1, Bullet, PyTorch 2.4.1, and CUDA 12.1 setup.\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotTrack\nconda env create -f environment.yml\nconda activate MiniCPM-RobotTrack\u003C\u002Fcode>\u003C\u002Fpre>\n\n#### 2. Prepare simulator data and assets\n\n\u003Cp>\n  Download HM3D, MP3D, humanoid, and robot assets following the\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fhabitat-sim\u002Fblob\u002Fmain\u002FDATASETS.md\">Habitat-Sim data instructions\u003C\u002Fa>\n  and \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fwsakobe\u002FTrackVLA\">TrackVLA asset instructions\u003C\u002Fa>.\n  Preserve their original directory structure under the project's \u003Ccode>data\u002F\u003C\u002Fcode> directory.\n\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode>data\u002F\n├── datasets\u002F\n├── scene_datasets\u002F\n├── humanoids\u002F\n└── robots\u002F\u003C\u002Fcode>\u003C\u002Fpre>\n\n### Data Preparation\n\n#### 1. Unprocessed rollouts\n\n\u003Cp>\n  Each Habitat rollout retains its video, per-step simulator state, and episode result.\n  The runnable \u003Ca href=\"MiniCPM-RobotTrack\u002Fsim_data\u002Fraw_sample\">\u003Ccode>sim_data\u002Fraw_sample\u003C\u002Fcode>\u003C\u002Fa>\n  example includes per-step records in the following format:\n\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-json\">{\n  \"base_velocity\": [0.62, 0.00, 0.02],\n  \"collision\": false,\n  \"target_distance\": 1.72,\n  \"human_center_norm\": [0.50, 0.48]\n}\u003C\u002Fcode>\u003C\u002Fpre>\n\n#### 2. Generate trajectory data\n\n\u003Cp>The command below keeps successful episodes, extracts RGB frames, and converts future actions into trajectory JSONL for finetuning:\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotTrack\npython tools\u002Fmake_tracking_data.py \\\n  --input_root sim_data\u002Fraw_sample \\\n  --output_root sim_data\u002Ftrain\u002Fstt \\\n  --only_success \\\n  --history 31 \\\n  --horizon 8 \\\n  --dt 0.1 \\\n  --incremental\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>For the full dataset, place the raw STT, DT, and AT rollouts under \u003Ccode>sim_data\u002Fraw\u002F<task>\u003C\u002Fcode> and run the same command for each task.\u003C\u002Fp>\n\n#### 3. Pre-cache visual features\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotTrack\nfor task in stt dt at; do\n  python tools\u002Fprecompute_features.py \\\n    --json \"sim_data\u002Ftrain\u002F${task}\u002Fjsonl\" \\\n    --data-root \"sim_data\u002Ftrain\u002F${task}\" \\\n    --cache-root \"sim_data\u002Ftrain\u002F${task}\u002Fvision_cache\"\ndone\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>\n  The processed outputs contain \u003Ccode>frames\u002F\u003C\u002Fcode>, \u003Ccode>jsonl\u002F\u003C\u002Fcode>, and\n  \u003Ccode>vision_cache\u002F\u003C\u002Fcode>.\n  \u003Ca href=\"MiniCPM-RobotTrack\u002Fsim_data\u002Fsample\">\u003Ccode>sim_data\u002Fsample\u003C\u002Fcode>\u003C\u002Fa>\n  provides two processed records for each of STT, DT, and AT, showing the final training-data format.\n\u003C\u002Fp>\n\n### Finetuning\n\n\u003Cp>After preparing data and visual caches for all three tasks, run the public finetuning entry point:\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotTrack\nbash scripts\u002Ftrain.sh\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>Data paths, batch size, learning rates, and epoch count can be adjusted in \u003Ccode>scripts\u002Ftrain.sh\u003C\u002Fcode>.\u003C\u002Fp>\n\n### Evaluation\n\n\u003Cp>\n  Download the complete released Hugging Face snapshot to\n  \u003Ccode>minicpm_robot_track\u002Fcheckpoints\u002FMiniCPM-RobotTrack\u002F\u003C\u002Fcode>. It contains the\n  fine-tuned MiniCPM4 backbone and is loaded with Transformers\n  \u003Ccode>from_pretrained\u003C\u002Fcode>. Evaluation still requires DINOv3 ViT-S\u002F16 and\n  SigLIP So400m; existing local vision-model directories can be selected with\n  environment variables:\n\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">export DINOV3_MODEL_PATH=\u002Fpath\u002Fto\u002Fdinov3-vits16\nexport SIGLIP_MODEL_PATH=\u002Fpath\u002Fto\u002Fsiglip-so400m-patch14-384\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>After the environment, models, and simulator assets are ready, run the evaluation script for each task:\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotTrack\nCKPT=minicpm_robot_track\u002Fcheckpoints\u002FMiniCPM-RobotTrack\nbash scripts\u002Feval_stt.sh \"$CKPT\" results\u002Fstt\nbash scripts\u002Feval_dt.sh  \"$CKPT\" results\u002Fdt\nbash scripts\u002Feval_at.sh  \"$CKPT\" results\u002Fat\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>Each command runs the evaluation for the corresponding task.\u003C\u002Fp>\n\n### Go2 Deployment\n\n\u003Cp>\n  The real-robot workflow targets a Unitree Go2 EDU with a Jetson Orin NX 16GB.\n  The validated stack is Jetson Linux R36.5, CUDA 12.6, TensorRT 10.7,\n  Python 3.10, ROS 2 Humble, and MAXN mode 0. Deployment uses separate Jetson\n  dependencies and defaults to \u003Ccode>dry-run\u003C\u002Fcode>, which never sends motion\n  commands.\n\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode>Go2\u002FD435i RGB -> TCP JPEG -> DINO + SigLIP TensorRT\n              -> MiniCPM-RobotTrack -> waypoint -> rate-limited control\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>\n  The complete setup uses a Go2 EDU, Orin NX 16GB, and D435i. The built-in Go2\n  front camera is the default validated source; each installation must validate\n  the D435i RGB path separately. Detailed hardware parameters, network settings,\n  asset download instructions, flashing instructions, and live-control procedures are kept\n  in the deployment documentation:\n\u003C\u002Fp>\n\n\u003Cul>\n  \u003Cli>\u003Ca href=\"MiniCPM-RobotTrack\u002Fdocs\u002FGO2_DEPLOYMENT.md\">Go2 deployment and reproduction guide\u003C\u002Fa>\u003C\u002Fli>\n  \u003Cli>\u003Ca href=\"MiniCPM-RobotTrack\u002Fdocs\u002FASSETS.md\">Model and deployment assets\u003C\u002Fa>\u003C\u002Fli>\n  \u003Cli>\u003Ca href=\"MiniCPM-RobotTrack\u002Fdocs\u002FJETPACK6_UPGRADE.md\">JetPack 6 upgrade guide\u003C\u002Fa>\u003C\u002Fli>\n\u003C\u002Ful>\n\n#### Quick Start\n\n\u003Cp>The following assumes JetPack 6 and the carrier-board patch are already installed:\u003C\u002Fp>\n\n\u003Cp>\n  Download the checkpoint manually from\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fopenbmb\u002FMiniCPM-RobotTrack\">openbmb\u002FMiniCPM-RobotTrack\u003C\u002Fa>.\n  Place the complete Hugging Face snapshot, including custom model code,\n  tokenizer files, and \u003Ccode>model.safetensors\u003C\u002Fcode>, in\n  \u003Ccode>MiniCPM-RobotTrack\u002Fminicpm_robot_track\u002Fcheckpoints\u002FMiniCPM-RobotTrack\u002F\u003C\u002Fcode>\n  before running preflight.\n\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotTrack\n\npython3 scripts\u002Fdownload_upstream_assets.py\n\npython3 -m pip install --user -r requirements-build.txt\n.\u002Fscripts\u002Fexport_onnx.sh\n\nsudo nvpmodel -m 0\nsudo jetson_clocks\n.\u002Fscripts\u002Fbuild_engines.sh\n\n.\u002Fscripts\u002Fpreflight.sh\n.\u002Fgo2_runtime.py run\u003C\u002Fcode>\u003C\u002Fpre>\n\n\u003Cp>Status and shutdown:\u003C\u002Fp>\n\n\u003Cpre>\u003Ccode class=\"language-bash\">cd MiniCPM-RobotTrack\n.\u002Fgo2_runtime.py status\n.\u002Fgo2_runtime.py stop-control\n.\u002Fgo2_runtime.py stop\u003C\u002Fcode>\u003C\u002Fpre>\n\n> **Safety:** Keep `runtime.mode: dry-run` for the first run. Before enabling\n> live control, validate the camera, model, latency, and stop path on a stand\n> with an on-site operator and a working remote\u002FApp emergency stop. Follow the\n> [Go2 deployment guide](MiniCPM-RobotTrack\u002Fdocs\u002FGO2_DEPLOYMENT.md) for the\n> complete live-control procedure and release limits.\n\n## Model Zoo\n\n| Model | Description | Download |\n| --- | --- | --- |\n| MiniCPM-RobotManip | A 1.5B vision-language-action model for Robot Manipulation | [🤗](https:\u002F\u002Fhuggingface.co\u002Fopenbmb\u002FMiniCPM-RobotManip) [\u003Cimg src=\"assets\u002Fmodelscope_logo.png\" height=\"12\" alt=\"ModelScope\">](https:\u002F\u002Fmodelscope.cn\u002Fmodels\u002FOpenBMB\u002FMiniCPM-RobotManip) |\n| MiniCPM-RobotTrack | A 0.9B vision-language-action model for Target Tracking | [🤗](https:\u002F\u002Fhuggingface.co\u002Fopenbmb\u002FMiniCPM-RobotTrack) [\u003Cimg src=\"assets\u002Fmodelscope_logo.png\" height=\"12\" alt=\"ModelScope\">](https:\u002F\u002Fmodelscope.cn\u002Fmodels\u002FOpenBMB\u002FMiniCPM-RobotTrack) |\n\n## License\n\nModel weights and code are open-sourced under the [Apache-2.0](.\u002FLICENSE) license.\n## Acknowledgments\n\n\u003Cp>\n  MiniCPM-Robot builds on and references MiniCPM,\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FstarVLA\u002FstarVLA\">starVLA\u003C\u002Fa>,\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Flerobot\">LeRobot\u003C\u002Fa>,\n  DINOv3, SigLIP, Habitat-Lab, Habitat-Sim, EVT-Bench, and TrackVLA.\n  We thank the authors and communities for their open-source contributions.\n  Third-party models, simulator code, datasets, and assets retain their own licenses; see\n  \u003Ca href=\"MiniCPM-RobotTrack\u002FTHIRD_PARTY_NOTICES.md\">THIRD_PARTY_NOTICES.md\u003C\u002Fa>.\n\u003C\u002Fp>\n",2,"2026-07-21 02:30:03","CREATED_QUERY"]