[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-78170":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":15,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":10,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":23,"hasPages":21,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":32,"readmeContent":33,"aiSummary":34,"trendingCount":16,"starSnapshotCount":16,"syncStatus":35,"lastSyncTime":36,"discoverSource":37},78170,"ConceptSeg-R1","NTU-AI4X\u002FConceptSeg-R1","NTU-AI4X","Segment Any Concept via Meta-Reinforcement Learning","",null,"Python",234,19,13,1,0,6,204,7,3.9,false,"main",true,[25,26,27,28,29,30,31],"concept-segmentation","generalized-concept-segmentation","image-segmentation","image-segmentation-pytorch","object-segmentation","unified-concept-segmentation","unified-image-segmentation","2026-06-12 02:03:46","\u003Cdiv align=\"center\">\n\n\u003Ch1>ConceptSeg-R1\u003C\u002Fh1>\n\n**ConceptSeg-R1: Segment Any Concept via Meta-Reinforcement Learning**\n\n[![arXiv](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2026-b31b1b?style=flat-square&logo=arxiv)](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2605.20385)\n[![Project Page](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🌐%20Project-Page-blueviolet?style=flat-square)](https:\u002F\u002Fntu-ai4x.github.io\u002FConceptSeg-R1\u002F)\n[![HuggingFace](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🤗%20Model-7B%20Weights-ffd21e?style=flat-square)](https:\u002F\u002Fhuggingface.co\u002Fzhaoyuan666\u002FConceptSeg-R1-7B)\n[![Dataset](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🤗%20Dataset-ConceptSeg--Benchmark-ffd21e?style=flat-square)](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fzhaoyuan666\u002FConceptSeg-Benchmark)\n[![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-Apache%202.0-blue?style=flat-square)](LICENSE)\n[![Stars](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002Fyuanzhao-CVLAB\u002FConceptSeg-R1?style=flat-square)](https:\u002F\u002Fgithub.com\u002Fyuanzhao-CVLAB\u002FConceptSeg-R1\u002Fstargazers)\n\n\u003Cp>\n  \u003Ca href=\"#introduction\">Introduction\u003C\u002Fa> •\n  \u003Ca href=\"#get-started\">Get Started\u003C\u002Fa> •\n  \u003Ca href=\"#data\">Data\u003C\u002Fa> •\n  \u003Ca href=\"#datasets--checkpoints\">Checkpoints\u003C\u002Fa>\n\u003C\u002Fp>\n\n\n\u003Cimg src=\".\u002Fassets\u002FConcept_Tree.png\" width=\"90%\"\u002F>\n\u003C\u002Fdiv>\n\n## 🎬 Short Video\n\u003Ca href=\"https:\u002F\u002Fntu-ai4x.github.io\u002FConceptSeg-R1\u002F#Show\">\n  \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002FNTU-AI4X\u002FNTU-AI4X.github.io\u002Fblob\u002Fmain\u002FConceptSeg-R1\u002FConceptSeg-R1-video.jpg\" width=\"90%\">\n\u003C\u002Fa>\n\n## 📰 News\n\n- **May 2026** — arXiv paper released 🎉\n\n## 🗺️ Roadmap\n\n| Status | Item |\n|:------:|------|\n| ✅ | arXiv paper |\n| ✅ | Training code |\n| ✅ | Testing code |\n| ✅ | CI-CD-CR datasets |\n| ✅ | ConceptSeg-R1 (7B weights) |\n| ⬜ | Support larger MLLM backbones, e.g., Gemini 2.5 Pro|\n\n\n## Introduction\n\n\u003Cdiv align=\"center\">\n\n### 🌍 As segmentation in computer vision shifts from objects to concepts, \n### 🚀 **ConceptSeg-R1 takes the first step toward segmenting any concept.**\n\n\u003C\u002Fdiv>\n\n\u003Cdiv align=\"center\">\n\u003Cimg src=\".\u002Fassets\u002FArchitecture.png\" width=\"100%\"\u002F>\n\u003C\u002Fdiv>\n\n\u003Cbr>\n\n### Key Contributions\n- **🌳 From Objects to Concepts**  \n  We introduce a three-level concept hierarchy covering **CI**, **CD**, and **CR** concepts, pushing segmentation beyond category recognition.\n\n- **🔁 From Instance Solving to Rule Induction**  \n  Meta-GRPO enables the model to infer transferable task rules from visual demonstrations and apply them deductively to unseen queries.\n\n- **🔗 Latent Concept Tokens for Frozen SAM 3**  \n  We map MLLM reasoning states into implicit concept tokens in the SAM 3 prompt space, enabling reasoning-aware segmentation without fine-tuning SAM 3.\n\n- **⚡ From Heavy Reasoning to Adaptive Inference**  \n  The Shortcut Router dynamically balances SAM 3 efficiency and reasoning depth, enabling fast perception for simple cases and deeper reasoning for complex concepts.\n\n## Results\n\n### Concept Segmentation Benchmarks (CI \u002F CD \u002F CR)\n\n\u003Cdiv align=\"center\">\n\u003Cimg src=\".\u002Fassets\u002Ftab1.png\" width=\"100%\"\u002F>\n\u003C\u002Fdiv>\n\u003Cbr>\n\n### Cityscapes Performance (Zero-Shot)\n\n\n\u003Cdiv align=\"center\">\n\u003Cimg src=\".\u002Fassets\u002Ftab2.png\" width=\"90%\"\u002F>\n\u003C\u002Fdiv>\n\u003Cbr>\n\n### ReasonSeg Performance (Zero-Shot)\n\n\n\u003Cdiv align=\"center\">\n\u003Cimg src=\".\u002Fassets\u002Ftab3.png\" width=\"60%\"\u002F>\n\u003C\u002Fdiv>\n\n### Qualitative Comparison\n\n\u003Cdiv align=\"center\">\n\u003Cimg src=\".\u002Fassets\u002Ffig4.png\" width=\"100%\"\u002F>\n\u003C\u002Fdiv>\n\u003Cbr>\n\n### Concept Coexistence\n\n\n\u003Cdiv align=\"center\">\n\u003Cimg src=\".\u002Fassets\u002Ffig5.png\" width=\"100%\"\u002F>\n\u003C\u002Fdiv>\n\u003Cbr>\n\n## Get Started\n\n### 1. Environment Setup\n\nBefore running `setup.sh`, download the release assets below from\n[GitHub Releases](https:\u002F\u002Fgithub.com\u002Fyuanzhao-CVLAB\u002FConceptSeg-R1\u002Freleases)\nand place them in the repository root:\n\n- `sam3-main.zip`: the modified SAM 3 package used by ConceptSeg-R1.\n- `all_meta.json.zip`: the training metadata file.\n\n```bash\nconda create -n conceptseg-r1 python=3.10\nconda activate conceptseg-r1\nbash setup.sh\n```\n\n### 2. Training\n\n**Prepare data** — Download the dataset, extract `all_meta.json` through `setup.sh`,\nand set your `image_folders` path in the shell scripts.\n\n```bash\n# Stage 1: SFT Training\nbash run_grpo_multiimage_stage1.sh\n\n# Stage 2: GRPO Training\nbash run_grpo_multiimage_stage2.sh\n```\n\n### 3. Evaluation\n\n**Concept Segmentation** — Download weights, set the model path in `eval_conceptseg.sh`, then run:\n\n```bash\nbash eval_conceptseg.sh\n```\n\n> **Tip:** Configure specific tasks for testing inside `eval_conceptseg.sh`.\n\n**Reasoning Segmentation** — Download weights, set the model path in `eval_reasonseg.sh`, then run:\n\n```bash\nbash eval_reasonseg.sh\n```\n\n## Data\n\n`all_meta.json` is no longer tracked in this repository. Download\n`all_meta.json.zip` from\n[GitHub Releases](https:\u002F\u002Fgithub.com\u002Fyuanzhao-CVLAB\u002FConceptSeg-R1\u002Freleases)\nand run `bash setup.sh` to extract it before training.\n\nPlace datasets under a shared root directory (`image_folders`):\n\n```\nroot\u002F\n├── isic2018\u002F\n├── rare\u002F\n├── Breast_Tumor\u002F\n├── transparent1024\u002F\n├── MGrounding-630k\u002F\n├── Polyp\u002F\n├── Shadow_detection\u002F\n├── MIG-Bench\u002F\n├── coco2014_Living\u002F\n├── CoSOD3k1024\u002F\n├── ultra_rare\u002F\n├── coco2014_Artifact\u002F\n├── fewshot1000\u002F\n├── DUTS\u002F\n├── ESDIDefects\u002F\n└── COD10K1024\u002F\n```\n\n\n## Metric\n\nEvaluation uses the [PySegMetric_EvalToolkit](https:\u002F\u002Fgithub.com\u002FXiaoqi-Zhao-DLUT\u002FPySegMetric_EvalToolkit).\n\n\n## Datasets & Checkpoints\n\n| Resource | Link |\n|----------|------|\n| 📦 ConceptSeg-Benchmark Dataset | [Download on HuggingFace](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fzhaoyuan666\u002FConceptSeg-Benchmark) |\n| 🤖 ConceptSeg-R1-7B Weights | [Download on HuggingFace](https:\u002F\u002Fhuggingface.co\u002Fzhaoyuan666\u002FConceptSeg-R1-7B) |\n","ConceptSeg-R1 是一个通过元强化学习实现任意概念分割的项目。其核心功能包括引入三层概念层次结构（CI、CD 和 CR），使得图像分割不仅限于对象识别，还扩展到更广泛的概念。该项目利用Meta-GRPO技术从视觉演示中推导出可迁移的任务规则，并将其应用于未见过的查询。此外，它还通过将多模态大语言模型的推理状态映射到SAM 3提示空间中的隐式概念令牌来实现无需微调SAM 3的推理感知分割。此项目的自适应推理机制能够根据任务复杂度动态调整计算资源分配，从而在保持高效的同时支持深度推理。ConceptSeg-R1适用于需要对复杂或抽象概念进行精准图像分割的应用场景，如自动驾驶、医疗影像分析等领域。",2,"2026-06-11 03:56:33","CREATED_QUERY"]