[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-106516":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":17,"stars90d":15,"forks30d":15,"starsTrendScore":18,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":21,"hasPages":21,"topics":23,"createdAt":10,"pushedAt":44,"updatedAt":45,"readmeContent":46,"aiSummary":10,"trendingCount":15,"starSnapshotCount":15,"syncStatus":47,"lastSyncTime":48,"discoverSource":49},106516,"HarborRL","datamllab\u002FHarborRL","datamllab","An end-to-end framework for Agentic Reinforcement Learning.","",null,"Python",86,8,5,0,10,24,44,77.26,"MIT License",false,"main",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"agent","agent-framework","agentic","agentic-ai","agentrl","coding-agent","coding-agents","dapo","grpo","harbor","harbor-framework","llm","llm-agents","llms","ppo","prm","reinforcement-learning","reinforcement-learning-agent","rl","slime","2026-10-30 07:13:21","2026-11-02 18:23:38","\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002FHarborRL-title.jpg\" alt=\"HarborRL\" width=800>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ch1>HarborRL\u003C\u002Fh1>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cstrong>A lightweight agentic reinforcement learning framework for everyone\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"pyproject.toml\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fversion-0.1.0-blue.svg\" alt=\"Version\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fwww.python.org\u002Fdownloads\u002F\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpython-3.10%2B-blue.svg\" alt=\"Python 3.10+\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fpytorch.org\u002F\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-2.4%2B-ee4c2c.svg\" alt=\"PyTorch 2.4+\">\u003C\u002Fa>\n  \u003Ca href=\"LICENSE\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-green.svg\" alt=\"MIT License\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  English · \u003Ca href=\"README_zh.md\">简体中文\u003C\u002Fa>\n\u003C\u002Fp>\n\n---\n\n## News\n\n- **2026-10-26** Added an [offline configuration wizard](docs\u002Fconfiguration-wizard.md) and a [local Web console](docs\u002Fweb-console.md) for configuration, validation, dry-runs, training launch, and logs.\n- **2026-10-26** Added [controller\u002FGPU deployment configuration](docs\u002Fdeployment.md), optional [OpenSandbox execution](docs\u002Fsandbox-rollout.md), and opt-in [train\u002Finference consistency checks](docs\u002Ftrain-inference-consistency.md).\n- **10\u002F2026** 📣📣📣 HarborRL released\n\n## About\n\nHarborRL is a lightweight framework for training tool-using agents on verifiable, sandboxed tasks. It makes model serving, trajectory generation, tool execution, verifier rewards, and policy updates into a single reproducible pipeline.\n\nHarborRL supports flexible combinations of **Models × Algorithms × Tasks × Harnesses (MATH)**:\n\n- **Models:** RL training using Qwen and GLM as base models.\n- **Algorithms:** GRPO and DAPO for RL training.\n- **Tasks:** RL training on **200+** Harbor-format tasks in isolated workers, including Terminal-Bench, SWE-Bench, Deep-SWE, and more.\n- **Harnesses:** RL training with **40+** harness frameworks, including Claude Code, Codex, LangGraph, SWE-agent, and more.\n\n## AI-assisted Configure\n\nFor the offline options catalog, interactive configuration, and answer-file setup,\nsee the [configuration CLI guide](docs\u002Fconfiguration-wizard.md).\nStart with `harborrl hello` for a CPU walkthrough requiring no model, GPU or task\nJSON. It executes scripted attempts and a real verifier, then exports a GRPO batch\nwith fixture tokens. Continue with `harborrl options` or `harborrl init --help`.\nThe wizard detects one shared Local environment, calculates model fingerprints,\nand supports incomplete drafts (`:save` \u002F `--resume`).\n\nRun `harborrl web` to open the local browser console for configuration, validation,\ndrafts, filled Local\u002FSandbox examples, Hello World, dry-runs, training launch, and logs. See the [web console guide](docs\u002Fweb-console.md)\nfor remote-controller access and lifecycle details.\n\nIf you want an AI agent to help deploy this repository and verify training,\nask it to first read and strictly follow\n[docs\u002Fagents\u002Fenvironment-check.md](docs\u002Fagents\u002Fenvironment-check.md).\n\nThe practiced workflow is to give the agent this prompt, replacing the bracketed\nvalues with your topology:\n\n```text\nRead docs\u002Fagents\u002Fenvironment-check.md and follow it exactly.\nTarget: [CPU development only | native split CPU-rollout\u002FGPU-training deployment]\nConfig: \u003CPath to config.yaml>, or none\nTopology: controller=\u003CYour IP address>:\u003CPort Number>;\n  rollout workers=\u003CRollout worker IP address>:\u003CPort Number>;\n  GPU machine=\u003CGPU machine IP address>:\u003CPort Number>\n\nFirst run read-only checks only. Return the required structured report and tell\nme the smallest next command that fixes one blocker. Do not install, download,\npull an image, mount storage, or start a service without my explicit consent.\nAfter each approved change, rerun the relevant checks. Finish with harborrl\ndoctor and launch_native --dry-run for a native deployment.\n```\n\nConfirm the result yourself against this checklist:\n\n1. The agent reports one acceptance level: `cpu_development`,\n   `deployment_preflight`, or `training`; it does not promote a CPU-only check\n   or dry run to a higher level.\n2. For CPU development, `uv run harborrl --help` and `uv run pytest` both pass.\n3. For native deployment, every statically verifiable rollout-worker, GPU, and\n   cross-machine check is `ready`; `harborrl doctor` has no failed checks; and\n   `uv run harborrl train \u003CPath to config.yaml> --dry-run` resolves the launch\n   plan.\n4. Every `unknown` item names what is deferred and why. Live gateway and\n   task-image checks can legitimately remain deferred before training.\n5. The agent waits for explicit consent before starting real training.\n\n## Quick Start\n\nThe default CPU install does not need a Sandbox SDK or cluster account.\nFor controller placement, see [deployment](docs\u002Fdeployment.md). For optional\nstandard OpenSandbox execution, see [Sandbox rollout](docs\u002Fsandbox-rollout.md).\nThe independently installed [H-cluster practice](practices\u002Fh_cluster\u002FREADME.md)\ncontains Brainbox-specific deployment and restricted-network relay examples.\n\n### Install\n\nUse Python 3.10 or newer. Runtime dependencies, development dependencies, and the `harborrl` command are locked by [pyproject.toml](pyproject.toml) and [uv.lock](uv.lock). The CPU development install does not require CUDA, Docker, model weights, Slime, Megatron-LM, or SGLang.\n\nInstall [uv](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002F) 0.9 or newer, then synchronize the locked CPU development environment:\n\n```bash\nuv sync --extra dev\n```\n\nExpected result: uv reads `uv.lock`, creates `.venv\u002F`, and installs HarborRL with `pytest` and `ruff`. The command does not install CUDA kernels, Docker, model weights, Slime, Megatron-LM, or SGLang.\n\nRun commands through the locked environment:\n\n```bash\nuv run harborrl --help\nuv run pytest\n```\n\nExpected result: uv executes the installed entry point and tests without activating the virtual environment manually. `train` accepts command composition or a YAML file; `doctor` validates a YAML deployment.\n\nFor native training, install the pinned backend stack in a writable location:\n\n```bash\nbash scripts\u002Fbootstrap_backends.sh \"\u003CPath to HarborRL backends>\"\n. \"\u003CPath to HarborRL backends>\u002Fharborrl-backend.env\"\n```\n\nThe generated file exports `SLIME_DIR`, `MEGATRON_DIR`, and `SGLANG_IMAGE` to child\nprocesses. For files generated by older bootstrap scripts, run `set -a` before\nsourcing and `set +a` afterward. See [rollout lifecycle and recovery](docs\u002Frollout-runtime-lifecycle.md)\nfor cleanup markers and CPU-only regression checks.\n\n### Train\n\nAfter installation, choose one of the two configuration styles:\n\n```bash\n# Compose the four experiment dimensions.\nuv run harborrl train -model qwen3-8B -harness claude-code -task terminal-bench -alg grpo\n\n# Load a complete, reproducible experiment definition.\nuv run harborrl train \u003CPath to config.yaml>\n```\n\nFor a bootstrap-aware YAML launch, use:\n\n```bash\nbash scripts\u002Flaunch_native.sh --config \u003CPath to config.yaml> --backends \u003CPath to HarborRL backends>\n```\n\nThe command style is a shorthand over `config.yaml` in the working directory (or the file selected with `--config`) and selects the four experiment dimensions. The YAML style is the complete launch definition: it also pins checkpoints, workers, serving settings, GPU topology, sampling limits, and output locations. Start from [examples\u002Fnative\u002Ftrain_qwen_native.yaml](examples\u002Fnative\u002Ftrain_qwen_native.yaml) and keep site-specific files outside the repository.\n\n## Task Configuration\n\nTraining is configured with one YAML file that has a fixed section and field set. Relative paths resolve against the YAML file, and repeated `--set section.field=value` overrides are parsed as YAML values and validated again. Start site-specific configurations from [examples\u002Fnative\u002Ftrain_qwen_native.yaml](examples\u002Fnative\u002Ftrain_qwen_native.yaml). Optional [train\u002Finference consistency checks](docs\u002Ftrain-inference-consistency.md) compare frozen rollout scores with the pre-update trainer and probe synchronized inference engines.\n\nThe configuration has three broad parts: `execution` is fixed to `harbor_job`; `harbor`, `gateway`, and `deployment` describe rollout workers, service endpoints, and GPU topology; `tasks`, `harness`, `model`, `training`, `sampling`, and `output` describe the experiment and its outputs.\n\nHarborRL consumes Harbor-recognized native tasks directly and does not maintain private conversion formats for individual benchmarks. A typical task directory is:\n\n```text\ntask\u002F\n├── task.toml\n├── instruction.md\n├── environment\u002FDockerfile\n└── tests\u002Ftest.sh\n```\n\nHere, `task.toml` declares resources, the agent, and the verifier; `instruction.md` is the agent-facing instruction; `environment\u002FDockerfile` builds the isolated environment; and `tests\u002Ftest.sh` verifies the result and writes `reward.json` or `reward.txt`. For training, tasks are locked into a catalog with their task ID, upstream revision, local path, content digest, and reward parsing rules; see [examples\u002Fnative\u002Fcatalog.json](examples\u002Fnative\u002Fcatalog.json).\n\nThe bundled runnable example is `hello_world`. It allows internet access so Harbor can install the pinned Claude Code CLI, reserves 4 GiB each for memory and storage, and writes a 0\u002F1 reward to `\u002Flogs\u002Fverifier\u002Freward.json`. Its profile allows four turns so the agent can observe a tool result and finish. Offline deployments need the exact CLI version preinstalled and a reachable policy gateway. After changing a task, regenerate its catalog digest.\n\nThe example pins the runtime-validated Claude Code `2.1.141`. The adapter disables prompt caching and cache warmup, compaction, and nonessential traffic, and always disallows `Agent` and `Task` even with a custom tool denylist. CLI `2.1.0` still makes an independent file-path-extraction model request after Bash and is incompatible with the single-context rollout contract. Probe other CLI versions before use; native session audit rejects auxiliary contexts, summaries, and unconsumed model responses.\n\nReal training tasks are not repackaged by this repository: fetch Harbor-native datasets or tasks from Harbor Hub, and use the [compatibility summary](configs\u002Fharbor_hub\u002Fpublic_compatibility_summary.md) to understand validated coverage. After fetching a task root, validate it locally with `harborrl hub inspect` and lock it into your own catalog.\n\n## License\n\nHarborRL is released under the MIT license.\n\n## Citation\n\nIf you use HarborRL for your research, please cite our technical report:\n```\n\n```\n\n## Acknowledgement\n\nWe thank [Slime](https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime) and [Harbor](https:\u002F\u002Fgithub.com\u002Fharbor-framework\u002Fharbor) for developing the foundational infrastructure for training and task execution.\n",2,"2026-11-03 04:32:10","CREATED_QUERY"]