[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72199":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":15,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":22,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":26,"readmeContent":27,"aiSummary":28,"trendingCount":16,"starSnapshotCount":16,"syncStatus":29,"lastSyncTime":30,"discoverSource":31},72199,"OpenManus-RL","OpenManus\u002FOpenManus-RL","OpenManus","A live stream development of RL tunning for LLM agents","",null,"Python",4096,575,57,15,0,5,10,28,30.28,"Apache License 2.0",false,"main",true,[],"2026-06-12 02:03:00","# OpenManus-RL\n🤗 \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FCharlieDreemur\u002FOpenManus-RL\" target=\"_blank\">Dataset (OpenManus-RL)\u003C\u002Fa>\n\nOpenManus-RL is an open-source initiative collaboratively led by __Ulab-UIUC__ and __MetaGPT__ .\n\nThis project is an extended version of the original [@OpenManus](https:\u002F\u002Fgithub.com\u002FFoundationAgents\u002FOpenManus) initiative. Inspired by successful RL tunning for reasoning LLM such as Deepseek-R1, QwQ-32B, we will explore new paradigms for RL-based LLM agent tuning, particularly building upon foundations.\n\nWe are committed to regularly updating our exploration directions and results in a dynamic, live-streaming fashion. All progress, including rigorous testing on agent benchmarks such as GAIA, AgentBench, WebShop, and OSWorld, and tuned models, will be openly shared and continuously updated.\n\nWe warmly welcome contributions from the broader community—join us in pushing the boundaries of agent reasoning and tool integration!\n\nCode and dataset are now available! The `verl` submodule has been integrated for enhanced RL training capabilities.\n\n\u003Cdiv style=\"display: flex; justify-content: center;\">\n  \u003Cdiv style=\"width: 100; transform: scale(1.0);\">\n    \u003Cimg src=\"assets\u002Fmanus.jpg\" style=\"width: 100%;\" alt=\"marble\">\n  \u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n## 📖 Table of Contents\n\n- [OpenManus-RL](#openmanus-rl)\n  - [🔔 News](#-news)\n  - [Current Team Members](#current-team-members)\n  - [How to Contribute](#how-to-contribute)\n  - [Roadmap](#roadmap)\n  - [Method](#method)\n    - [Reasoning Models Exploration](#reasoning-models-exploration)\n    - [Alternative Rollout Strategies](#alternative-rollout-strategies)\n    - [Environment and Benchmark](#environment-and-benchmark)\n    - [Post-Training Strategies](#post-training-strategies)\n    - [Training of Agent Reward Model](#training-of-agent-reward-model)\n    - [Test-time Scaling of Trajectories](#test-time-scaling-of-trajectories)\n    - [Action Space Awareness and Strategic Exploration](#action-space-awareness-and-strategic-exploration)\n    - [Integration with RL Tuning Frameworks](#integration-with-rl-tuning-frameworks)\n  - [Dataset](#dataset)\n    - [Dataset Overbiew](#dataset-overview)\n    - [Data Instances](#data-instances)\n- [Running](#Running)\n- [Related Work](#related-work)\n  - [Agent tuning](#agent-tuning)\n  - [Tool using](#tool-using)\n  - [Agent tuning instruction dataset](#agent-tuning-instruction-dataset)\n  - [RL tuning](#rl-tuning)\n  - [Benchmark](#benchmark)\n  - [Similar Code](#similar-code)\n- [Acknowledgement](#acknowledgement)\n- [Community Group](#community-group)\n- [Citation](#citation)\n- [Documentation](#documentation)\n\n---\n\n\n## 🔔 News\n- **[2025-03-09]** 🍺 We collect and opensource our Agent SFT dataset at [Huggingface](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FCharlieDreemur\u002FOpenManus-RL), go try it!\n- **[2025-03-08]** 🎉 We are collaborating with [@OpenManus](https:\u002F\u002Fgithub.com\u002Fmannaandpoem\u002FOpenManus) from Metagpt to work on this project together!\n- **[2025-03-06]** 🥳 We(UIUC-Ulab) are announcing our live-streaming project, OpenManus-RL.\n\n\n## Current Team Members\n[@Kunlun Zhu](https:\u002F\u002Fgithub.com\u002FKunlun-Zhu)(Ulab-UIUC), [@Muxin Tian](https:\u002F\u002Fgithub.com\u002Frealtmxi), [@Zijia Liu](https:\u002F\u002Fm-serious.github.io\u002F)(Ulab-UIUC), [@Yingxuan Yang](https:\u002F\u002Fgithub.com\u002Fzoe-yyx),[@Jiayi Zhang](https:\u002F\u002Fgithub.com\u002Fdidiforgithub)(MetaGPT), [@Xinbing Liang](https:\u002F\u002Fgithub.com\u002Fmannaandpoem), [@Weijia Zhang](https:\u002F\u002Fgithub.com\u002FCharlieDreemur), [@Haofei Yu](https:\u002F\u002Fgithub.com\u002Flwaekfjlk)(Ulab-UIUC), [@Cheng Qian](https:\u002F\u002Fqiancheng0.github.io\u002F),[@Bowen Jin](https:\u002F\u002Fgithub.com\u002FPeterGriffinJin), \n\n---\n\n# How to Contribute\nWe wholeheartedly welcome suggestions, feedback, and contributions from the community! Feel free to:\n\nWe welcome contributions, including fine-tuning codebase, tuning dataset, environment setup, and computing resources.\nCreate issues for feature requests, bug reports, or ideas.\nSubmit pull requests to help improve OpenManus-RL.\nOr simply reach out to us for direct collaboration.\nImportant contributors will be listed as co-authors to our paper.\n\n# Roadmap\n1. Agent Environment Support\nSetting up LLM agent environment for online RL tunning.\n\n2. Agent Trajectories Data Collection\nConnect to specialized reasoning models such as deepseek-r1, QwQ-32B for more complex inference tasks to collect comprehensive agent trajectories.\n\n3. RL-Tuning Model Paradigm\nProvide an RL fine-tuning approach for customizing the agent's behavior in our agent environment.\n\n4. Test on Agent Benchmarks\nEvaluate our framework on agentic benchmark such as Webshop, GAIA, OSWorld, AgentBench\n\n\n\n\u003Cdiv style=\"display: flex; justify-content: center;\">\n  \u003Cdiv style=\"width: 100; transform: scale(1.0);\">\n    \u003Cimg src=\"assets\u002Fopenmanus-roadmap.png\" style=\"width: 100%;\" alt=\"marble\">\n  \u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n## Method\n\nOur method proposes an advanced reinforcement learning (RL)-based agent tuning framework designed to significantly enhance reasoning and decision-making capabilities of large language models (LLMs). Drawing inspiration from RAGEN's Reasoning-Interaction Chain Optimization (RICO), our approach further explores novel algorithmic structures, diverse reasoning paradigms, sophisticated reward strategies, and extensive benchmark environments.\n\n### Reasoning Models Exploration\nTo benchmark the reasoning capabilities effectively, we evaluate multiple state-of-the-art reasoning models:\n- **GPT-O1**\n- **Deepseek-R1**\n- **QwQ-32B**\n\nEach model provides unique reasoning capabilities that inform downstream optimization and training strategies.\n\n### Alternative Rollout Strategies\nWe experiment with a variety of rollout strategies to enhance agent planning efficiency and reasoning robustness, including:\n\n- **Tree-of-Thoughts (ToT)**: Employs tree-based reasoning paths, enabling agents to explore branching possibilities systematically.\n- **Graph-of-Thoughts (GoT)**: Utilizes graph structures to represent complex reasoning dependencies effectively.\n- **DFSDT (Depth-First Search Decision Trees)**: Optimizes action selection through depth-first search, enhancing long-horizon planning.\n- **Monte Carlo Tree Search (MCTS)**: Explores reasoning and decision paths probabilistically, balancing exploration and exploitation effectively.\n\nThese methods help identify optimal rollout techniques for various reasoning tasks.\n\n### Diverse Reasoning Formats\nWe specifically analyze and compare several reasoning output formats, notably:\n\n- **ReAct**: Integrates reasoning and action explicitly, encouraging structured decision-making.\n- **Outcome-based Reasoning**: Optimizes toward explicit outcome predictions, driving focused goal alignment.\n\nThese formats are rigorously compared to derive the most effective reasoning representation for various tasks.\n\n### Post-Training Strategies\nWe investigate multiple post-training methodologies to fine-tune agent reasoning effectively:\n\n- **Supervised Fine-Tuning (SFT)**: Initializes reasoning capabilities using human-annotated instructions.\n- **Generalized Reward-based Policy Optimization (GRPO)**: Incorporates:\n    - **Format-based Rewards**: Rewards adherence to specified reasoning structures.\n    - **Outcome-based Rewards**: Rewards accurate task completion and goal attainment.\n- **Proximal Policy Optimization (PPO)**: Enhances agent stability through proximal updates.\n- **Direct Preference Optimization (DPO)**: Leverages explicit human preferences to optimize agent outputs directly.\n- **Preference-based Reward Modeling (PRM)**: Uses learned reward functions derived from human preference data.\n\n### Training of Agent Reward Model\nWe train specialized agent reward models using annotated data to accurately quantify nuanced reward signals. These models are then leveraged to guide agent trajectory selection during both training and evaluation phases.\n\n### Test-time Scaling of Trajectories\nDuring the inference phase, trajectory scaling methods are implemented, allowing agents to flexibly adapt to varying task complexities, thus enhancing robustness and performance in real-world scenarios.\n\n### Action Space Awareness and Strategic Exploration\nAgents are equipped with action-space awareness, employing systematic exploration strategies designed to navigate complex action spaces effectively, ultimately maximizing expected rewards.\n\n### Integration with RL Tuning Frameworks\nWe integrate insights and methodologies from leading RL tuning frameworks, including:\n\n- **Verl** - **Integrated as Git Submodule** - Our primary RL framework, providing advanced training capabilities for agent optimization\n- **TinyZero**\n- **OpenR1**\n- **Trlx**\n\n### Verl Integration\nThe `verl` submodule is fully integrated into OpenManus-RL, providing:\n- **Advanced RL Algorithms** - PPO, DPO, and custom reward modeling\n- **Efficient Training** - Optimized for large language model fine-tuning\n- **Flexible Configuration** - Easy customization of training parameters\n- **Production Ready** - Battle-tested framework from Bytedance\n\nThrough these frameworks, agents can effectively balance exploration and exploitation, optimize reasoning processes, and adapt dynamically to novel environments.\n\nIn summary, our method systematically integrates advanced reasoning paradigms, diverse rollout strategies, sophisticated reward modeling, and robust RL frameworks, significantly advancing the capability and adaptability of reasoning-enhanced LLM agents.\n\n\u003Cdiv style=\"display: flex; justify-content: center;\">\n  \u003Cdiv style=\"width: 100; transform: scale(1.0);\">\n    \u003Cimg src=\"assets\u002Fmethod_overview.png\" style=\"width: 100%;\" alt=\"marble\">\n  \u003C\u002Fdiv>\n\u003C\u002Fdiv>\n\n# Dataset\n[**OpenManusRL-Dataset**](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FCharlieDreemur\u002FOpenManus-RL) combines agent trajectories from [AgentInstruct](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FTHUDM\u002FAgentInstruct), [Agent-FLAN](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Finternlm\u002FAgent-FLAN) and [AgentTraj-L(AgentGym)] with features:\n\n- 🔍 **ReAct Framework** - \u003Ca href=\"https:\u002F\u002Freact-lm.github.io\u002F\" target=\"_blank\">Reasoning-Acting integration\u003C\u002Fa>\n- 🧠 **Structured Training** - Separate format\u002Freasoning learning\n- 🚫 **Anti-Hallucination** - Negative samples + environment grounding\n- 🌐 **6 Domains** - OS, DB, Web, KG, Household, E-commerce\n\n### Dataset Overview\n| Source | Trajectories | Avg Turns | Key Features |\n|--------|--------------|-----------|--------------|\n| [AgentInstruct](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FTHUDM\u002FAgentInstruct) | 1,866 | 5.24 | Multi-task QA, CoT reasoning |\n| [Agent-FLAN](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Finternlm\u002FAgent-FLAN) | 34442 | 3-35 | Error recovery patterns, diverse real-world tasks|\n  [AgentTraj-L](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FAgentGym\u002FAgentTraj-L) | 14485 | 3-35 | interactive environments and tasks\n| **Combined** | 50793 | 4-20 | Enhanced generalization, uniform format with broader task coverage |\n\n### Supported Tasks\n- **text-generation**: ReAct-style instruction following\n- **conversational-ai**: Tool-augmented dialogues\n\n### Languages\nEnglish\n\n## Data Instances\n\n**ReAct Pattern Example**:\n```json\n{\n  \"id\": \"os_0\",\n  \"conversations\": [\n    {\"role\": \"user\", \"content\": \"Count files in \u002Fetc\"},\n    {\"role\": \"assistant\", \"content\": \"Think: Need reliable counting method\\nAct: bash\\n```bash\\nls -1 \u002Fetc | wc -l\\n```\"},\n    {\"role\": \"user\", \"content\": \"OS Output: 220\"},\n    {\"role\": \"assistant\", \"content\": \"Think: Verified through execution\\nAct: answer(220)\"}\n  ]\n}\n```\n\n# Running\n\n## OpenManus-RL\n\nA simplified library for Supervised Fine-Tuning (SFT) and GRPO tunning of language models for agentic system. (developed upon [Verl](https:\u002F\u002Fgithub.com\u002Fvolcengine\u002Fverl) from Bytedance)\nWe are still laboriously developing this part, welcome feedback.\n\n## Installation\n\n### Prerequisites\nThis project uses git submodules. After cloning the repository, make sure to initialize and update the submodules:\n\n```bash\n# Clone the repository with submodules\ngit clone --recursive https:\u002F\u002Fgithub.com\u002FOpenManus\u002FOpenManus-RL.git\n\n# Or if already cloned, initialize and update submodules\ngit submodule update --init --recursive\n```\n\n### Environment Setup\nFirst, create a conda environment and activate it:\n\n```bash\n# Create a new conda environment\nconda create -n openmanus-rl python=3.10 -y\nconda activate openmanus-rl\n```\n\nThen, install the required dependencies:\n\n```bash\n# Install PyTorch with CUDA support\npip3 install torch torchvision\n\n# Install vllm for efficient inference\n# Install the main package\npip install -e .[vllm]\n\n# flash attention 2\npip3 install flash-attn --no-build-isolation\npip install wandb\n\n```\n\n## Environment Setup\n\n### 1. Webshop\nTo set up the WebShop environment for evaluation:\n\n```bash\n# Change to the agentenv-webshop directory\ncd openmanus_rl\u002Fenvironments\u002Fenv_package\u002Fwebshop\u002Fwebshop\u002F\n\n# Create a new conda environment for WebShop\nconda create -n agentenv_webshop python==3.10 -y\nconda activate agentenv_webshop\n\n# Setup the environment\nbash .\u002Fsetup.sh -d all\n```\n\n### 2. ALFWorld\n\n```bash\nconda acitvate openmanus-rl\npip3 install gymnasium==0.29.1\npip3 install stable-baselines3==2.6.0\npip install alfworld\n```\n\nDownload PDDL & Game files and pre-trained MskRCNN detector (will be stored in `~\u002F.cache\u002Falfworld\u002F`):\n```\nalfworld-download -f\n```\nUse `--extra` to download pre-trained checkpoints and seq2seq data.\n\n## Quick Start\n\n### 1. Environment Setup\nMake sure you have the required environments set up (see Environment Setup section above).\n\n### 2. Data Preparation\nDownload the OpenManus-RL dataset from [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FCharlieDreemur\u002FOpenManus-RL).\n\n### 3. Training Examples\n\n#### ALFWorld RL Training (PPO)\n```bash\nconda activate openmanus-rl\nbash scripts\u002Fppo_train\u002Ftrain_alfworld.sh\n```\n\n\n\n\n# Related Work\n\n## Agent tuning\n\n1. **Offline Training of Language Model Agents with Functions as Learnable Weights**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2402.11359)]\n2. **FIREACT : TOWARD LANGUAGE AGENT FINE-TUNING**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2310.05915)]\n3. **AgentTuning: Enabling Generalized Agent Abilities for LLMs**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2310.12823)]\n4. **ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2403.14589)]\n5. **UI-TARS: Pioneering Automated GUI Interaction with Native Agents**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2501.12326#page=16.83)]\n6. **ATLAS: Agent Tuning via Learning Critical Steps**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2503.02197)]\n\n## Tool using\n\n1. **Toolformer: Language Models Can Teach Themselves to Use Tools**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2302.04761)]\n2. **ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs**. [[paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2307.16789)]\n\n## Agent tuning instruction dataset\n\n1. **Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2403.12881)]\n2. **AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2402.15506)]\n\n## RL tuning\n\n1. **Training Language Models to Follow Instructions with Human Feedback**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2305.18438)]\n2. **Deepseekmath: Pushing the Limits of Mathematical Reasoning in Open Language Models**. [[paper](https:\u002F\u002Fproceedings.neurips.cc\u002Fpaper_files\u002Fpaper\u002F2022\u002Ffile\u002Fb1efde53be364a73914f58805a001731-Paper-Conference.pdf)]\n3. **DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning**. [[paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2501.12948)]\n\n## Benchmark:\n\n1. **AgentBench: Evaluating LLMs as Agents**. [paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2308.03688)\n2. **WebShop: Towards Scalable Real-World Web Interaction with Autonomous Agents**. [paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2207.01206)\n3. **GAIA: a benchmark for General AI Assistants**. [paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2311.12983)\n4. **ALFWorld: Aligning Text and Embodied Environments for Interactive Learning**. [paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2010.03768)\n\n## Similar framework\n\n1. **RAGEN: Training Agents by Reinforcing Reasoning**. [[code](https:\u002F\u002Fgithub.com\u002FZihanWang314\u002FRAGEN)]\n2. **verl-agent**. [[code](https:\u002F\u002Fgithub.com\u002FlangfengQ\u002Fverl-agent)]\n\n## Offline RL\n1. **D4RL: Datasets for Deep Data-Drive Reinforcement Learning**. [[paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2004.07219)]\n2. **Offline Reforcement Learning with Implicit Q-Learning**. [[paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2110.06169)]\n3. **Behavior Proximal Policy Optimization**. [[paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2302.11312)]\n\n# Acknowledgement\nWe extend our thanks to ulab-uiuc (https:\u002F\u002Fulab-uiuc.github.io\u002F) and Openmanus (https:\u002F\u002Fgithub.com\u002Fmannaandpoem\u002FOpenManus)) team from MetaGPT for their support and shared knowledge. Their mission and community contributions help drive innovations like OpenManus forward.\n\nWe also want to gratefully thank Verl (https:\u002F\u002Fgithub.com\u002Fvolcengine\u002Fverl) and verl-agent(https:\u002F\u002Fgithub.com\u002FlangfengQ\u002Fverl-agent) for their opensource.\n\nWe welcome all developers who are interested in this project can reach out to (kunlunz2@illinois.edu)\n\nStay tuned for updates and the official release of our repository. Together, let's build a thriving open-source agent ecosystem!\n\n# Community Group\n\nJoin our networking group on Feishu and share your experience with other developers!\n\n\u003Cdiv align=\"center\" style=\"display: flex; gap: 20px;\">\n    \u003Cimg src=\"assets\u002Fcommunity_group.jpg\" alt=\"OpenManus-RL 交流群\" width=\"300\" \u002F>\n\u003C\u002Fdiv>\n\n# Citation\nPlease cite the following paper if you find OpenManus helpful!\n```bibtex\n@misc{OpenManus,\n  author       = {OpenManus-RL Team},\n  title        = {OpenManus-RL: Open Platform for Generalist LLM Reasoning Agents with RL optimization},\n  year         = {2025},\n  organization = {GitHub},\n  url          = {https:\u002F\u002Fgithub.com\u002FOpenManus\u002FOpenManus-RL},\n}\n```\n\n\u003Cp align=\"center\">\n\u003Ca href=\"https:\u002F\u002Fstar-history.com\u002F#Significant-Gravitas\u002FAutoGPT\">\n  \u003Cpicture>\n    \u003Csource media=\"(prefers-color-scheme: dark)\" srcset=\"https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=OpenManus\u002FOpenManus-RL&type=Date&theme=dark\" \u002F>\n    \u003Csource media=\"(prefers-color-scheme: light)\" srcset=\"https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=OpenManus\u002FOpenManus-RL&type=Date\" \u002F>\n    \u003Cimg alt=\"Star History Chart\" src=\"https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=Significant-Gravitas\u002FAutoGPT&type=Date\" \u002F>\n  \u003C\u002Fpicture>\n\u003C\u002Fa>\n\u003C\u002Fp>\n\n## Project Structure\n\n```\nOpenManus-RL\u002F\n├── verl\u002F                    # Verl RL framework submodule\n├── openmanus_rl\u002F           # Main OpenManus-RL library\n├── scripts\u002F                # Training and evaluation scripts\n├── configs\u002F                # Configuration files\n├── environments\u002F           # Agent environment implementations\n├── docs\u002F                   # Documentation\n└── examples\u002F               # Usage examples\n```\n\n## Documentation\n- [Development Guide (English)](docs\u002FDEVELOPMENT_GUIDE_EN.md)\n- [Development Guide (Chinese)](docs\u002FDEVELOPMENT_GUIDE_ZH.md)\n- [Training Process Overview (English)](docs\u002FREADME.md)\n- [Training Process Overview (Chinese)](docs\u002FREADME_ZH.md)\n","OpenManus-RL 是一个用于大型语言模型（LLM）代理的强化学习调优的开源项目。该项目由 Ulab-UIUC 和 MetaGPT 共同领导，旨在探索基于强化学习的 LLM 代理调优新范式，特别是在推理能力方面。它集成了 verl 子模块以增强训练能力，并通过 GAIA、AgentBench、WebShop 和 OSWorld 等基准测试对代理进行严格评估。所有进展和调优模型均公开分享并持续更新。OpenManus-RL 适合需要提升 LLM 代理推理能力和工具集成的研究者和开发者使用。",2,"2026-06-11 03:40:49","high_star"]