[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72402":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":32,"readmeContent":33,"aiSummary":34,"trendingCount":16,"starSnapshotCount":16,"syncStatus":17,"lastSyncTime":35,"discoverSource":36},72402,"OM1","OpenMind\u002FOM1","OpenMind","Modular AI runtime for robots","https:\u002F\u002Fopenmind.com",null,"Go",2809,992,58,35,0,2,8,26,6,30.99,"MIT License",false,"main",true,[27,28,29,30,31],"llm","multiagent","robotics","ros2","zenoh","2026-06-12 02:03:02","![om1_banner_w](https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F86590615-018d-4443-b345-8d224227e83f)\n\n\u003Cp align=\"center\">\n\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2412.18588\">Technical Paper\u003C\u002Fa> |\n\u003Ca href=\"https:\u002F\u002Fdocs.openmind.com\u002F\">Documentation\u003C\u002Fa> |\n\u003Ca href=\"https:\u002F\u002Fx.com\u002Fopenmind_agi\">X\u003C\u002Fa>\n\u003C\u002Fp>\n\n**OpenMind's OM1 is a modular AI runtime that empowers developers to create and deploy multimodal AI agents across digital environments and physical robots**, including Humanoids, Phone Apps, Quadrupeds, educational robots such as TurtleBot 4, and simulators like Gazebo and Isaac Sim. OM1 agents can process diverse inputs like web data, social media, camera feeds, and LIDAR, while enabling physical actions including motion, autonomous navigation, and natural conversations. The goal of OM1 is to make it easy to create highly capable human-focused robots, that are easy to upgrade and (re)configure to accommodate different physical form factors.\n\n## Capabilities of OM1\n\n* **Modular Architecture**: Designed with Python for simplicity and seamless integration.\n* **Data Input**: Easily handles new data and sensors.\n* **Hardware Support via Plugins**: Supports new hardware through plugins for API endpoints and specific robot hardware connections to `ROS2`, `Zenoh`, and `CycloneDDS`. (We recommend `Zenoh` for all new development).\n* **Web-Based Debugging Display**: Monitor runtime state in WebSim (available at http:\u002F\u002Flocalhost:8000\u002F), including user input, and current move\u002Fspeech\u002Femotion outputs.\n* **Pre-configured Endpoints**: Supports Text-to-Speech, multiple LLMs from OpenAI, xAI, DeepSeek, Anthropic, Meta, Gemini, NearAI, Ollama (local), and multiple Visual Language Models (VLMs) with pre-configured endpoints for each service.\n* **Metrics & Observability**: Includes a pre-configured Prometheus and Grafana stack to monitor real-time AI pipeline metrics like LLM and ASR latencies.\n\n## Architecture Overview\n![Artboard 1@4x 1 (1)](https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F0c482257-e4db-4a0a-8d83-d4548ac4beaf)\n\n## Getting Started\n\nIf you are new to OM1, this is the fastest path to a successful first run using the `spot` agent.\n\nSpot uses your webcam to detect objects and sends those observations to the LLM. The model then returns move\u002Fspeak\u002Femotion outputs, which are visualized in WebSim for debugging.\n\nSpot in this quick start is the default starter configuration to help you understand the OM1 pipeline and WebSim output. WebSim visualizes state updates and does not execute robot hardware actions.\n\n### Quick Start (5 Minutes)\n\n1. Install system dependencies.\n2. Clone the repository.\n3. Add your OpenMind API key.\n4. Launch OM1 and verify WebSim output.\n\n### Prerequisites\n\n- Python 3.10+\n- [`uv` package manager](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002Fgetting-started\u002Finstallation\u002F)\n- Webcam access (recommended if configuring VLM)\n\nInstall system packages:\n\n### 1. Install System Dependencies\n\nFor macOS:\n```bash\nbrew install portaudio ffmpeg\n```\n\nFor Linux:\n```bash\nsudo apt-get update\nsudo apt-get install -y portaudio19-dev python3-dev ffmpeg\n```\n\n### 2. Clone\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FOpenMind\u002FOM1.git\ncd OM1\ngit submodule update --init\nuv venv\n```\n\n### 3. Configure API Key\n\nGet your API key from [OpenMind Portal](https:\u002F\u002Fportal.openmind.com\u002F).\n\n1. Sign in to OpenMind Portal.\n2. Open the dashboard and create a new API key.\n3. Copy the generated key.\n\nRecommended (shell profile):\n```bash\nexport OM_API_KEY=\"\u003Cyour_api_key>\"\n```\n\nAlternative (project-local):\n```bash\ncp .env.example .env\n```\n\nThen set:\n```bash\nOM_API_KEY=\u003Cyour_api_key>\n```\nin `.env`.\n\nYou can also verify or adjust the fallback key location in `config\u002Fspot.json5`.\n\n### 4. Launch Spot\n\n```bash\nuv run src\u002Frun.py spot\n```\n\n#### Verify It Is Working\n\nOpen \u003Chttp:\u002F\u002Flocalhost:8000\u002F> in your browser.\n\nYour setup is successful if:\n\n- The terminal indicates that WebSim started.\n- The WebSim page loads at port `8000`.\n- You can see Spot state updates (inputs, move\u002Fspeak\u002Femotion outputs, and latency) in the WebSim UI.\n\n### 5. Monitor with Grafana (Optional)\n\nIf you have Docker installed, you can launch the included Prometheus and Grafana stack to monitor real-time AI pipeline metrics such as LLM and ASR latencies:\n\n```bash\ndocker-compose up -d grafana prometheus\n```\n\nNavigate to \u003Chttp:\u002F\u002Flocalhost:3000> in your browser (default login: `admin`\u002F`admin`). The **OM1 Latency Monitoring** dashboard is automatically provisioned and ready to use.\n\n### Troubleshooting\n\n- `Authentication` errors: confirm `OM_API_KEY` is set and not expired.\n- `No module` errors: run the command with `uv run` from the repo root.\n- `Camera` access issues: grant terminal\u002FIDE camera permissions in OS settings.\n- `Address already in use` on port `8000`: stop the conflicting process or free the port.\n\n### OMCU\n\nOMCU is the computational unit for billing on OpenMind's platform. The free plan provides 50 OMCU renewed monthly.\n\nUpgrade your plan [here](https:\u002F\u002Fportal.openmind.com\u002F) for additional credits.\n\nFor more help connecting OM1 to your robot hardware, see [getting started](https:\u002F\u002Fdocs.openmind.com\u002Fdeveloping\u002F1_get-started).\n\n> **Note:** This quick start uses the Spot starter configuration. WebSim is a runtime visualization and debugging interface, not a hardware control endpoint. For voice interactions, ensure ASR and TTS are configured in `config\u002Fspot.json5`.\n\n## What's Next?\n\n* Try out some [examples](https:\u002F\u002Fdocs.openmind.com\u002Fdeveloper-cookbook\u002Fexamples)\n* Add new `inputs` and `actions`.\n* Design custom agents and robots by creating your own `json5` config files with custom combinations of inputs and actions.\n* Change the system prompts in the configuration files (located in `\u002Fconfig\u002F`) to create new behaviors.\n\n## Interfacing with New Robot Hardware\n\nOM1 assumes that robot hardware provides a high-level SDK that accepts elemental movement and action commands such as `backflip`, `run`, `gently pick up the red apple`, `move(0.37, 0, 0)`, and `smile`. An example is provided in `src\u002Factions\u002Fmove\u002Fconnector\u002Fros2.py`:\n\n```python\n...\nelif output_interface.action == \"shake paw\":\n    if self.sport_client:\n        self.sport_client.Hello()\n...\n```\n\nIf your robot hardware does not yet provide a suitable HAL (hardware abstraction layer), traditional robotics approaches such as RL (reinforcement learning) in concert with suitable simulation environments (Unity, Gazebo), sensors (such as hand mounted ZED depth cameras), and custom VLAs will be needed for you to create one. It is further assumed that your HAL accepts motion trajectories, provides battery and thermal management\u002Fmonitoring, and calibrates and tunes sensors such as IMUs, LIDARs, and magnetometers.\n\nOM1 can interface with your HAL via USB, serial, ROS2, CycloneDDS, Zenoh, or websockets. For an example of an advanced humanoid HAL, see [Unitree's C++ SDK](https:\u002F\u002Fgithub.com\u002Funitreerobotics\u002Funitree_sdk2\u002Fblob\u002Fadee312b081c656ecd0bb4e936eed96325546296\u002Fexample\u002Fg1\u002Fhigh_level\u002Fg1_loco_client_example.cpp#L159). Frequently, a HAL, especially ROS2 code, is dockerized and interfaces with OM1 through DDS middleware or websockets.\n\n## Recommended Development Platforms\n\nOM1 is developed on:\n\n* Nvidia Thor (running JetPack 7.0) - full support\n* Jetson AGX Orin 64GB (running Ubuntu 22.04 and JetPack 6.1) - limited support\n* Mac Studio with Apple M2 Ultra with 48 GB unified memory (running MacOS Sequoia)\n* Mac Mini with Apple M4 Pro with 48 GB unified memory (running MacOS Sequoia)\n* Generic Linux machines (running Ubuntu 22.04)\n\nOM1 _should_ run on other platforms (such as Windows) and microcontrollers such as the Raspberry Pi 5 16GB.\n\n## Introduction to BrainPack\n\nFrom research to real-world autonomy, a platform that learns, moves, and builds with you.\n\nThe BrainPack is designed to be mounted directly onto a robot to bring together mapping, object recognition, remote control, and self charging, giving humanoids and quadrupeds what they need to navigate, remember, and act with purpose.\n\n## Full Autonomy Guidance\n\nOM1 supports **full autonomy** for Unitree Go2 and G1 with BrainPack. The following features are supported with BrainPack:\n\n- **Navigation** - Autonomous path planning and movement.\n- **SLAM** - Simultaneous Localization and Mapping for persistent map-based operation.\n- **Auto Charging** - Automated docking and battery charging workflows.\n- **Face Detection and Anonymization** - Real-time perception and privacy-aware processing.\n\nFor more details, see [Full Autonomy](docs\u002Ffull_autonomy_guidelines\u002Farchitecture_overview.md).\n\nThe BrainPack is open-source and you can refer to the guidelines to build your own [here](https:\u002F\u002Fgithub.com\u002FOpenMind\u002Fbrainpack).\n\n## Simulator Support\n\nOM1 integrates with popular robotics simulators to enable rapid prototyping and testing without physical hardware.\n\n### Gazebo\n\nOpen source Gazebo support is designed for rapid prototyping, conversational interaction, and behavior testing. Gazebo integration with OM1 is supported for Unitree Go2.\n\nSee [Gazebo](docs\u002Fsimulators\u002Fgazebo.md) to get started.\n\n### Isaac Sim\n\nNVIDIA Isaac Sim support enables physics-accurate simulation with GPU acceleration. Isaac Sim integration with OM1 is supported for Unitree Go2 and G1.\n\nRequires NVIDIA GPU and CUDA support. See [Isaac Sim Setup](docs\u002Fsimulators\u002Fisaac-sim.md) to get started.\n\n## Detailed Documentation\n\nMore detailed documentation can be accessed at [docs.openmind.com](https:\u002F\u002Fdocs.openmind.com\u002F).\n\n## Contributing\n\nPlease make sure to read the [Contributing Guide](.\u002FCONTRIBUTING.md) before making a pull request.\n\n## License\n\nThis project is licensed under the terms of the MIT License, which is a permissive free software license that allows users to freely use, modify, and distribute the software. The MIT License is a widely used and well-established license that is known for its simplicity and flexibility. By using the MIT License, this project aims to encourage collaboration, modification, and distribution of the software.\n","OpenMind的OM1是一个模块化AI运行时，旨在帮助开发者创建和部署多模态AI代理，适用于数字环境和实体机器人，如人形机器人、手机应用、四足机器人及教育机器人等。其核心功能包括处理网络数据、社交媒体、摄像头流和激光雷达信息，并支持物理动作如运动控制、自主导航和自然对话。OM1采用Python开发，具有高度的可扩展性和简易集成性，通过插件支持新硬件接入ROS2、Zenoh等系统。此外，OMOM1还提供了基于Web的调试显示界面WebSim以及预配置的Prometheus和Grafana监控栈，便于开发者实时监控AI管道性能。该项目特别适合需要快速构建并迭代具备高级交互能力机器人的场景。","2026-06-11 03:41:53","high_star"]