[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-80712":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":13,"subscribersCount":13,"size":13,"stars1d":13,"stars7d":14,"stars30d":14,"stars90d":13,"forks30d":13,"starsTrendScore":13,"compositeScore":13,"rankGlobal":10,"rankLanguage":10,"license":10,"archived":16,"fork":16,"defaultBranch":17,"hasWiki":16,"hasPages":16,"topics":18,"createdAt":10,"pushedAt":10,"updatedAt":19,"readmeContent":20,"aiSummary":21,"trendingCount":13,"starSnapshotCount":13,"syncStatus":15,"lastSyncTime":22,"discoverSource":23},80712,"GoLongRL","xiaoxuanNLP\u002FGoLongRL","xiaoxuanNLP","GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment","",null,"Python",45,0,1,2,false,"main",[],"2026-06-12 02:04:05","# 🎯 GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment\n\n\u003Cdiv align=\"center\">\n\n[![Paper](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPaper-arXiv-b31b1b.svg)](https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.19577)\n[![Paper](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPaper-Hugging%20Face-ffd21e?logo=huggingface)](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2605.19577)\n[![Model](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FModel-GoLongRL--4B-2c74d7?logo=huggingface)](https:\u002F\u002Fhuggingface.co\u002FKwai-Klear\u002FGoLongRL-4B)\n[![Model](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FModel-GoLongRL--30B--A3B-2c74d7?logo=huggingface)](https:\u002F\u002Fhuggingface.co\u002FKwai-Klear\u002FGoLongRL-30B-A3B)\n[![Dataset](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%20Dataset-GoLongRL-2c74d7?logo=huggingface)](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FKwai-Klear\u002FGoLongRL)\n[![Contact](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F📧%20Contact-Email-blue)](mailto:xiao_xuan_zi_666@163.com)\n\n\u003C\u002Fdiv>\n\n## 📣 Latest News\n\n**[May 19, 2026]**  📢 GoLongRL is available on [arXiv](https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.19577) and [HuggingFace Daily](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2605.19577).\n\n## 📌 Overview\n\nWe present **GoLongRL**, a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). \n\nExisting long-context RL methods tend to focus data construction on retrieval-path complexity—multi-hop chains, UUID tracking, chunk-based QA—while collapsing diverse task objectives into a single binary reward. This leaves critical capabilities such as summarization, ranking, aggregation, and structured reasoning without direct training signal. GoLongRL addresses these limitations through two contributions.\n\nGoLongRL addresses these limitations through two key contributions. **(1) Capability-oriented data construction**, a 23K-sample RLVR dataset spanning 9 task types with heterogeneous reward functions, and **(2) TMN-Reweight**, a multitask optimization method that combines task-level mean normalization for cross-task reward scale alignment with difficulty-adaptive reweighting for more reliable advantage estimation.\n\n\u003Cdiv align=\"center\">\n\n\u003Cimg src=\".\u002Fassets\u002Fmain_result.png\" width=\"100%\"\u002F>\n\n\u003Csub>Overall performance comparison on long-context benchmarks (DocMath, LongBench-V2, Frames, MRCR, CorpusQA, LBV1-QA).\u003C\u002Fsub>\n\n\u003C\u002Fdiv>\n\n**GoLongRL-30B-A3B** achieves long-context performance comparable to DeepSeek-R1-0528 and Qwen3-235B-A22B-Thinking-2507, while using a significantly smaller activated parameter budget.\n\n| Model | Avg. | DocMath | LBV2 | Frames | MRCR | CorpusQA | LBV1-QA |\n|---|---|---|---|---|---|---|---|\n| Qwen3-30B-A3B-Thinking-2507 | 60.1 | 63.3 | 48.7 | 70.2 | 41.6 | 70.5 | 66.5|\n| DeepSeek-R1-0528 | 68.7 | 63.4 | 59.5 | 76.9 | 64.9 | 77.5 | 69.9 |\n| Qwen3-235B-A22B-Thinking | 68.5 | 65.8 | 57.5 | 75.1 | 66.2 | 75.3 | 70.9 |\n| Gemini-2.5-Flash-Thinking | 68.7 | 64.8 | 56.8 | 65.8 | 78.8 | 79.4 | 66.9 |\n| QwenLong-L1.5 (w. GRPO) | 67.2 | 65.1 | 55.3 | 71.4 | 66.9 | 76.9 | 67.9 |\n| **GoLongRL-30B-A3B (Ours)** | **69.8** | **65.3** | **55.1** | **74.5** | **81.6** | **73.6** | **68.7** |\n\nOur framework combines the following.\n\n1. **Capability-Oriented Dataset (23K samples, 9 task types).** Guided by a taxonomy of long-context capabilities, the dataset covers precise retrieval, comprehension, exhaustive retrieval, numerical reasoning, structured extraction, structured matching, graded ranking, sequence ordering, and summarization. Each task is paired with its natural evaluation metric (EM, Accuracy, F1, math_verify, IoU, SubEM, NDCG, Pairwise, ROUGE-L) as the reward function, rather than being collapsed into a single indicator.\n\n2. **TMN-Reweight.** When training on heterogeneous reward types, per-prompt normalization in standard GRPO can mix up cross-task scale differences with prompt difficulty. TMN-Reweight is a simple modification that normalizes advantages at the task level instead of the prompt level, and adds a difficulty-adaptive weight to reduce noise from very easy or very hard prompts. It provides a modest but consistent improvement over vanilla GRPO in our ablations (+0.8 avg. at 4B scale), with gains mainly on aggregation-intensive benchmarks like CorpusQA.\n\n3. **Full Open Release.** We publicly release the complete dataset and all training code.\n\n### Key Results\n\n- Under the same vanilla GRPO setup, our dataset alone outperforms the closed-source QwenLong-L1.5 dataset at both 4B and 30B scales (+6.1 avg. at 4B, +2.6 avg. at 30B).\n- TMN-Reweight further improves average performance to 63.0 at 4B scale, surpassing QwenLong-L1.5 with its specialized AEPO algorithm (59.4).\n- General capabilities (MMLU-Pro, AIME24\u002F25, GPQA) are preserved or improved, with substantial gains on dialogue memory (LongMemEval +13.6) and agentic memory benchmarks.\n\n\u003Cdiv align=\"center\">\n\n\u003Cimg src=\".\u002Fassets\u002Fpipeline.png\" width=\"100%\"\u002F>\n\n\u003Csub>Overview of the four-phase Capability-Oriented RLVR Dataset construction pipeline.\u003C\u002Fsub>\n\n\u003C\u002Fdiv>\n\n---\n\n## 🧪 Training\n\nTraining is built on [verl](https:\u002F\u002Fgithub.com\u002Fvolcengine\u002Fverl) with two RL algorithms:\n\n- **GRPO** — Standard Group Relative Policy Optimization with per-group advantage normalization.\n- **TMN-GRPO** (Task-Mixed Normalization GRPO) — Normalizes advantages within **reward-type groups** rather than globally, preventing high-variance tasks from dominating gradients when training on a mixture of heterogeneous long-context tasks. Supports optional **difficulty reweighting** that up-weights hard prompts based on smoothed per-prompt pass rate.\n\nSupported models: **Qwen3-4B**, **Qwen3-30B-A3B** (MoE). Training runs on 16 nodes × 8 GPUs with SGLang async serving.\n\n### 📦 Training Data\n\nThe capability-oriented training set (23K samples, 9 task types) is publicly available:\n\n**Dataset**: [Kwai-Klear\u002FGoLongRL](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FKwai-Klear\u002FGoLongRL)\n\nFor full setup, data format, hyperparameters, and monitoring details, see [Train\u002Fverl\u002FREADME.md](Train\u002Fverl\u002FREADME.md).\n\n```bash\nexport LLM=\u002Fpath\u002Fto\u002FQwen3-4B\nexport TRAIN_FILE=\u002Fpath\u002Fto\u002Ftrain.jsonl\nexport TEST_FILE=\u002Fpath\u002Fto\u002Ftest.jsonl\n\nbash examples\u002FGoLongRL\u002Fqwen3-4B-grpo.sh           # GRPO baseline\nbash examples\u002FGoLongRL\u002Fqwen3-4B-tmn-reweight.sh   # TMN-GRPO + difficulty reweighting\n```\n\n---\n\n## 🔍 Evaluation\n\nEvaluation uses **QwenLong-Benchmarks**, covering three capability dimensions:\n\n| Dimension | Benchmarks |\n|-----------|-----------|\n| **Long-Context** | LongBench-V2, MRCR (≤128K \u002F 128K–512K \u002F 512K–1M), Frames, LongBench QA, DocMath, CorpusQA (≤128K \u002F ≤1M) |\n| **General** | MMLU-Pro, AIME 2024\u002F2025, GPQA-Diamond |\n| **Memory** | BFCL-V4 (memory subset), LongMemEval |\n\nUltra-long evaluations (up to 1M tokens) use YaRN RoPE scaling. For full setup and per-benchmark instructions, see [Eval\u002FQwenLong-Benchmarks\u002FREADME.md](Eval\u002FQwenLong-Benchmarks\u002FREADME.md).\n\n```bash\nconda create -n evalscope python=3.10 && conda activate evalscope\npip install -e Eval\u002FQwenLong-Benchmarks\u002Fevalscope && pip install vllm\n\nexport MODEL_PATH=\u002Fpath\u002Fto\u002Fmodel\nexport MODEL_NAME=your-model-name\nbash Eval\u002FQwenLong-Benchmarks\u002Fevalscope\u002Feval_mrcr.128K.sh\n```\n\n---\n\n## 🤝 Citation\n\n```\n@misc{lv2026golongrlcapabilityorientedlongcontext,\n      title={GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment}, \n      author={Minxuan Lv and Tiehua Mei and Tanlong Du and Junmin Chen and Zhenpeng Su and Ziyang Chen and Ziqi Wang and Zhennan Wu and Ruotong Pan and jian Liang and Ruiming Tang and Han Li},\n      year={2026},\n      eprint={2605.19577},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.19577}, \n}\n```\n","GoLongRL是一个面向能力的长上下文强化学习项目，通过多任务对齐技术提升模型在多种任务上的表现。其核心功能包括能力导向的数据构建和TMN-Reweight优化方法，前者提供了涵盖9种不同类型任务的23K样本数据集，后者则通过跨任务奖励尺度对齐与难度自适应重加权来提高优势估计的可靠性。该项目适用于需要处理复杂、长文本信息并执行如摘要生成、排名、聚合及结构化推理等多样化任务的场景。尽管规模较小，但GoLongRL在多个基准测试中展现出了与更大模型相媲美的性能。","2026-06-11 04:01:43","CREATED_QUERY"]