[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-93369":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":9,"totalLinesOfCode":9,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":9,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":17,"stars30d":17,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":18,"rankGlobal":9,"rankLanguage":9,"license":9,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":19,"hasPages":19,"topics":9,"createdAt":9,"pushedAt":9,"updatedAt":21,"readmeContent":22,"aiSummary":23,"trendingCount":16,"starSnapshotCount":16,"syncStatus":24,"lastSyncTime":25,"discoverSource":26},93369,"prime-rl","PrimeIntellect-ai\u002Fprime-rl","PrimeIntellect-ai","Agentic RL Training at Scale",null,"https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl","Python",1678,356,9,31,0,14,61.06,false,"main","2026-07-22 04:02:09","\u003Cp align=\"center\">\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F40c36e38-c5bd-4c5a-9cb3-f7b902cd155d#gh-light-mode-only\" alt=\"Prime Intellect\" width=\"312\">\n  \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F6414bc9b-126b-41ca-9307-9e982430cde8#gh-dark-mode-only\"  alt=\"Prime Intellect\" width=\"312\">\n\u003C\u002Fp>\n\n---\n\n\u003Ch3 align=\"center\">\nPRIME-RL: Async RL Training at Scale\n\u003C\u002Fh3>\n\n---\n\n\u003C\u002Fbr>\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Factions\u002Fworkflows\u002Fstyle.yaml\">\n    \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Factions\u002Fworkflows\u002Fstyle.yaml\u002Fbadge.svg\" alt=\"Style\" \u002F>\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Factions\u002Fworkflows\u002Fcpu_tests.yaml\">\n    \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Factions\u002Fworkflows\u002Fcpu_tests.yaml\u002Fbadge.svg\" alt=\"Test\" \u002F>\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Factions\u002Fworkflows\u002Fgpu_tests.yaml\">\n    \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Factions\u002Fworkflows\u002Fgpu_tests.yaml\u002Fbadge.svg\" alt=\"Test\" \u002F>\n  \u003C\u002Fa>\n\u003C\u002Fp>\n\n## Overview\n\nPRIME-RL is a framework for large-scale reinforcement learning. It is designed to be easy to use and hackable, yet capable of scaling to 1000+ GPUs. Here is what we think sets it apart:\n\n1. Fully asynchronous RL for high-throughput agentic training at scale.\n2. Performant: built to train 1T+ MoE models on 1000+ GPUs with [FSDP2](https:\u002F\u002Fdocs.pytorch.org\u002Ftutorials\u002Fintermediate\u002FFSDP_tutorial.html) for training and [vLLM](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm) for inference, with FP8 inference, PD disaggregation, EP and CP parallelism, and more.\n3. Native integration with [`verifiers`](https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fverifiers) environments through the [Environments Hub](https:\u002F\u002Fapp.primeintellect.ai\u002Fdashboard\u002Fenvironments?ex_sort=most_stars), including built-in support for SWE and agentic environments.\n4. End-to-end post-training: SFT, RL training, and evals.\n5. Multi-node deployment with Slurm and Kubernetes support.\n6. Multimodal support for VLMs such as Qwen3-VL.\n7. Hackable, modular, and extensible by design.\n8. One-line SLURM deployment for frontier models — e.g. [`GLM-5` FP8 with P\u002FD disaggregation, the `llm-d` router, and Mooncake KV offload](examples\u002Fglm5_llmd\u002FREADME.md).\n\n\n## Models support\n\n\nThe trainer works with both Hugging Face and Prime custom `ModelForCausalLM` out of the box. For selected families (especially large MoE) we also ship highly optimized training code under `src\u002Fprime_rl\u002Ftrainer\u002Fmodels\u002F`, including expert parallelism (EP) for MoE layers and context parallelism (CP) for long sequences (see the table), and additional kernels like [quack-kernels](https:\u002F\u002Fgithub.com\u002Fquack-kernels\u002Fquack-kernels).\n\nWith `[model] impl = \"auto\"` (the default), the trainer selects that custom stack when the Hugging Face config type is registered.\n\n| Family | Example IDs | MoE | EP | CP |\n|--------|-------------|-----|----|-----|\n| GLM-5 (`glm_moe_dsa`) | `zai-org\u002FGLM-5`, `zai-org\u002FGLM-5-FP8` | yes | ✅ | ✅ |\n| Qwen3 MoE (`qwen3_moe`) | `Qwen\u002FQwen3-30B-A3B`, … | yes | ✅ | ✅ |\n| Qwen3.5 MoE (`qwen3_5_moe`) | `Qwen\u002FQwen3.5-35B-A3B`, … | yes | ✅ | ✅ |\n| Qwen3 \u002F Qwen3.5 VLMs | see [advanced.md](docs\u002Fadvanced.md#vision-language-models) (`qwen3_vl`, `qwen3_5`, `qwen3_5_moe`) | MoE only on MoE VLMs | MoE only | ✅ |\n| Poolside Laguna (`laguna`) | `poolside\u002FLaguna-XS.2` | yes | ✅ | ✅ |\n| MiniMax M2 (`minimax_m2`) | `MiniMax\u002FMiniMax-M2` | yes | ✅ | ✅ |\n| Nemotron H (`nemotron_h`) | `nvidia\u002FNemotron-3-Nano-30B-A3B`, `nvidia\u002FNemotron-3-Super-120B-A12B`, … | yes | ✅ | ✅ |\n| Trinity (`afmoe`) | `arcee-ai\u002FTrinity-Mini`, … | yes | ✅ | ✅ |\n| GLM-4 · GLM-4.5 MoE · INTELLECT-3 (`glm4_moe`) | `THUDM\u002FGLM-4-9B-0414`, `zai-org\u002FGLM-4.5-Air`, `zai-org\u002FGLM-4.5`, `PrimeIntellect\u002FINTELLECT-3`, … | yes | ✅ | ✅ |\n| GPT-OSS (HF, MoE) | `openai\u002Fgpt-oss-20b`, `openai\u002Fgpt-oss-120b` | yes |  ✅ | ✅ |\n| Other HF causal LMs | Qwen3 dense, Mistral, … (`impl = \"hf\"`) | varies | ❌ | ✅ |\n\n\n## Setup\n\n> *We develop and test on NVIDIA RTX 3090\u002F4090\u002F5090, A100, H100, H200, and B200. If your setup fails, please create an [issue](https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Fissues).*\n\n### Prerequisites\n\nCurrently, you **need at least one NVIDIA GPU to use PRIME-RL**. If you don't already have access to one, we recommend our [compute platform](https:\u002F\u002Fapp.primeintellect.ai) for everything from renting on-demand single GPUs for developing, debugging and small ablations, to [reserving 1000+ GPU clusters](https:\u002F\u002Fapp.primeintellect.ai\u002Fdashboard\u002Fquotes) for production-scale training.\n\n### Quick Setup\n\nSet up PRIME-RL in a single command.\n\n```bash\ncurl -sSL https:\u002F\u002Fraw.githubusercontent.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Fmain\u002Fscripts\u002Finstall.sh | bash\n```\n\n\u003Cdetails>\n\u003Csummary>\nManual Setup\n\u003C\u002Fsummary>\n\u003Cbr>\n\n1. Clone the repository\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl.git\ncd prime-rl\n```\n\n2. Initialize submodules\n\n```bash\ngit submodule update --init -- deps\u002Fverifiers deps\u002Frenderers deps\u002Fresearch-environments deps\u002Fpydantic-config\n```\n\n3. Install [uv](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002F)\n\n```bash\ncurl -LsSf https:\u002F\u002Fastral.sh\u002Fuv\u002Finstall.sh | sh\nsource $HOME\u002F.local\u002Fbin\u002Fenv\n```\n\n4. Install dependencies from the lock file\n\n```bash\nuv sync --all-extras\n```\n\n> *NOTE*: Environments are opt-in uv workspace members — `uv sync --all-extras` does not install them. To train on them, install all with `uv sync --all-extras --all-packages`, or a subset with `uv sync --package prime-rl --package \u003Cenv>`.\n\n4.1. On aarch64 hosts: build flash-attn from source for your GPU\n\n> *NOTE*: aarch64 has no prebuilt flash-attn wheel. This step compiles the CUDA extension for your local GPU (~20-30 minutes). Compute capability is auto-detected from `nvidia-smi`; override with `TORCH_CUDA_ARCH_LIST=9.0` (Hopper) \u002F `10.0` (Blackwell) if needed.\n> *NOTE*: After this step, you can't run `uv sync --all-extras` or `uv run` as it will uninstall the package, you can avoid it by running `uv sync --inexact` or `uv run --no-sync`.\n\n```bash\nbash scripts\u002Fdocker-arm64-post-install.sh\n```\n\n3.1. Optional: Install Flash Attention 3 (on Hopper GPUs only, for flash_attention_3 attention backend)\n\n> *NOTE*: This step will take a while, as it builds the Flash Attention 3 extension from source, as it has no wheels prebuilt.\n> *NOTE*: After this step, you can't run `uv sync --all-extras` or `uv run` as it will uninstall the package, you can avoid it by running `uv sync --inexact` or `uv run --no-sync`\n\n```bash\nuv pip install \"flash-attn-3 @ git+https:\u002F\u002Fgithub.com\u002FDao-AILab\u002Fflash-attention.git@main#subdirectory=hopper\" --no-build-isolation\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\nValidate your environment setup\n\u003C\u002Fsummary>\n\u003Cbr>\n\n1. Check that the environment uses Python 3.12\n\n```bash\nuv run python -V\n```\n\n2. Check that `flash-attn` is installed\n\n```bash\nuv run python -c \"import flash_attn\"\n```\n\n3. Check that you can run SFT trainer  (*this requires 1 GPU*)\n\n```bash\nuv run sft @ configs\u002Fdebug\u002Fsft\u002Ftrain.toml\n```\n\n4. Check that you can run the RL trainer (*this requires 1 GPU*)\n\n```bash\nuv run trainer @ configs\u002Fdebug\u002Frl\u002Ftrain.toml\n```\n\n5. Check that you can run the inference server (*this requires 1 GPU*)\n\n```bash\nuv run inference @ configs\u002Fdebug\u002Finfer.toml\n```\n\n*Keep the inference server running in the background for the next steps.*\n\n5.1. Check that you can run the orchestrator against the inference server\n\n```bash\nuv run orchestrator @ configs\u002Fdebug\u002Forch.toml\n```\n\n5.2. Check that you can run evals against the inference server\n\n```bash\nuv run eval @ configs\u002Fdebug\u002Feval.toml\n```\n\n\u003C\u002Fdetails>\n\n### Additional Setup\n\n1. If you want to log your runs to [W&B](https:\u002F\u002Fwandb.ai), log in\n\n```bash\nuv run wandb login\n# Or set `export WANDB_API_KEY=...`\n```\n\n2. If you require gated\u002F private models or datasets from [HuggingFace](https:\u002F\u002Fhuggingface.co), log in\n\n```bash\nuv run hf auth login\n# Or set `export HF_TOKEN=...`\n```\n\n## Training Examples\nWe provide end-to-end training examples in the [`examples`](examples) directory to highlight features of the framework and guide you through the process of training your own models.\n\n### Basic Training: 1 to 8 GPUs\n\nFollow this guide to learn the basics of Prime-RL. You can train your own models on 1 to 8 GPUs. Ideal for getting started and exploring the capabilities of the framework. These guides cover most use cases -- single-turn, multi-turn, tool calling, etc. -- on toy environments and small models.\n\n1. [**Reverse Text**](examples\u002Freverse_text\u002FREADME.md): Train `Qwen3-0.6B` to reverse a small chunk of text. Demonstrates tiny-scale single-turn SFT and RL training. Can be trained on a single consumer GPU in a few minutes, and is ideal for getting started.\n2. [**Wordle**](examples\u002Fwordle\u002FREADME.md): Train `Qwen3-1.7B` to play Wordle. A fun example of multi-turn SFT and RL training. Can be trained on a 2-4 H100 GPUs in a few hours. Ideal for exploring the multi-turn training capabilities of the framework.\n3. [**Alphabet Sort**](examples\u002Falphabet_sort\u002FREADME.md): Train `Qwen3-4B-Instruct-2507` to sort names alphabetically. Demonstrates multi-turn RL training via LoRA without SFT warmup. Can be trained on a single H100 GPU in just over an hour. Ideal for exploring LoRA-based training.\n4. [**Wiki Search**](examples\u002Fwiki_search\u002FREADME.md): Train `Qwen3-4B-Instruct-2507` to answer trivia questions by searching through a Wikipedia. Demonstrates multi-turn with web search tool use.\n5. [**Hendrycks Sanity**](examples\u002Fhendrycks_sanity\u002FREADME.md): Run a sanity check experiment on `DeepSeek-R1-Distill-Qwen-1.5B` using a filtered subset of MATH where the model already partially solves 20-80% of problems. Useful for algorithm ablations.\n\n### Advanced Training: 32 - 2048 GPUs:\n\nFollow this guide to train large models on hard reasoning and agentic \u002F swe environments.\nThese guides are designed to be run from a Slurm cluster but can also be adapted to k8s deployments.\n\n1. [**Qwen 3 30B - A3B Math**](examples\u002Fqwen30b_math\u002FREADME.md): Train `Qwen3-30B-A3B` to solve hard math problems.\n2. [**Qwen 3 30B - A3B SWE**](examples\u002Fqwen30b_swe\u002FREADME.md): Train `Qwen3-30B-A3B` to solve hard SWE problems.\n3. [**Intellect-3.1**](examples\u002FIntellect-3.1\u002FREADME.md): Reproduce our `INTELLECT-3.1` training run.\n4. [**MiniMax-M2.5 SWE**](examples\u002Fminimax_m2.5_swe\u002FREADME.md): Train `MiniMax-M2.5` on agentic SWE tasks.\n5. [**High-throughput GLM-5**](examples\u002Fglm5_pd_disag\u002FREADME.md): Train `GLM-5` with PD disaggregation and FP8 inference on SWE.\n6. [**High-throughput GLM-5 (llm-d)**](examples\u002Fglm5_llmd\u002FREADME.md): One-line SLURM deployment for `GLM-5` FP8 with P\u002FD disaggregation, the `llm-d` router, and Mooncake KV offload — the faster way to run `GLM-5`.\n\n## Docs\n\nCheck out the [docs](docs) directory for in-depth guides on how to use PRIME-RL.\n\n- [**Overview**](docs\u002Foverview.md) - Architecture, install, and a copy-pasteable end-to-end RL run\n- [**Configuration**](docs\u002Fconfiguration.md) - TOML composition, CLI overrides, env vars, validation\n- [**Training**](docs\u002Ftraining.md) - RL, SFT, evals, checkpointing, observability, rules of thumb\n- [**Scaling**](docs\u002Fscaling.md) - Single-GPU through multi-node, FSDP\u002FEP\u002FCP, SLURM, benchmarking\n- [**Algorithms**](docs\u002Falgorithms.md) - Async\u002Foff-policy training, the AIPO loss, advantage and filter plugins, trajectory merging\n- [**Advanced**](docs\u002Fadvanced.md) - Custom modeling, multimodal training, LoRA, multi-tenant training\n- [**Development**](docs\u002Fdevelopment.md) - Test suite, pre-commit hooks, adding a new model\n\n## Contributing\n\nWe warmly welcome community contributions! We use [issues](https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl\u002Fissues) to track bugs, feature requests, and share our internal roadmap. If you encounter bugs, have pain points during development, or have ideas for new features, please open an issue.\n\nContributions are welcome via PR. Please follow these guidelines:\n1. Install the [pre-commit hooks](#pre-commit-hooks) to ensure your code is formatted correctly.\n2. Please keep your PR in \"Draft\" until it is ready for review.\n3. If your PR resolves an issue, please link the issue in the PR description\n4. If you can, try running the [test suite](#tests) locally to ensure your changes are working as expected.\n\n### Pre-Commit Hooks\n\nPlease install the [pre-commit](https:\u002F\u002Fpre-commit.com) hooks to ensure your code is formatted correctly.\n\n```bash\nuv run pre-commit install\n```\n\n### Tests\n\n```bash\nuv run pytest -v                    # everything\nuv run pytest tests\u002Funit -v         # unit only\nuv run pytest tests\u002Fintegration -v  # integration only\nuv run pytest -v -m \"not gpu\"       # CPU-only (inverse of the gpu marker)\n```\n\n## License\n\nThis project is licensed under the Apache 2.0 license, as found in the [License](LICENSE) file.\n\n## Citation\n\nIf you find our work useful, feel free to cite it using\n\n```tex\n@misc{primeintellect2025prime-rl,\n  author = {Prime Intellect},\n  title = {PRIME-RL},\n  url = {https:\u002F\u002Fgithub.com\u002FPrimeIntellect-ai\u002Fprime-rl},\n  year = {2025}\n}\n```\n","PRIME-RL 是一个面向大规模智能体强化学习（Agentic RL）训练的开源框架，支持千卡级 GPU 异步分布式训练。其核心特点包括：基于 FSDP2 和 vLLM 的高性能训练与推理栈，原生支持 MoE 模型的专家并行（EP）与上下文并行（CP），内置 SWE 等 agentic 环境集成，以及端到端的 SFT、RLHF 和评估流水线。适用于需要高吞吐、低延迟的智能体行为建模场景，如代码生成代理、多步决策系统和复杂任务自动化训练。",2,"2026-07-17 02:30:06","trending"]