[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92578":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":14,"stars30d":15,"stars90d":14,"forks30d":14,"starsTrendScore":14,"compositeScore":16,"rankGlobal":9,"rankLanguage":9,"license":17,"archived":18,"fork":18,"defaultBranch":19,"hasWiki":18,"hasPages":20,"topics":21,"createdAt":9,"pushedAt":9,"updatedAt":22,"readmeContent":23,"aiSummary":24,"trendingCount":14,"starSnapshotCount":14,"syncStatus":25,"lastSyncTime":26,"discoverSource":27},92578,"SkillOpt-Lite","EvolvingLMMs-Lab\u002FSkillOpt-Lite","EvolvingLMMs-Lab","SkillOpt-Lite and HarnessOpt: Optimize your skill or harness with one line of vibe",null,"Python",75,4,55,0,20,44.1,"MIT License",false,"main",true,[],"2026-07-22 04:02:06","# SkillOptLite and HarnessOpt: Better and Faster Agent Self-evolution with One Line of Vibe\n\n*Train your agent's skills and harness with a single line of vibe — automatic, looped improvements.*\n\n[![arXiv](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-coming%20soon-b31b1b)](#)\n[![Poster](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPoster-PDF-8a2be2)](images\u002Ficml2026_poster.pdf)\n[![Report](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FReport-PDF-informational)](images\u002Fskillopt_lite.pdf)\n[![HuggingFace Dataset](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🤗%20Dataset-SkillOpt__Lite__Benchmarks-yellow)](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fyshenaw\u002FSkillOpt_Lite_Benchmarks)\n[![Python 3.10+](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPython-3.10%2B-blue.svg)](https:\u002F\u002Fwww.python.org\u002F)\n[![License: MIT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-yellow.svg)](LICENSE)\n\n\u003Cp align=\"center\">\n👉 \u003Cb>\u003Ca href=\"https:\u002F\u002Fevolvinglmms-lab.github.io\u002FSkillOpt-Lite\u002F\">Project Page\u003C\u002Fa>\u003C\u002Fb> 👈\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n👉 \u003Cb>\u003Ca href=\"images\u002Fwechat-group.jpg\">WeChat Group\u003C\u002Fa>\u003C\u002Fb> 👈\n\u003C\u002Fp>\n\n---\n\n## TL;DR\n\n**Open this repo folder inside your coding agent** (VS Code Copilot\nChat, Codex CLI, Claude Code, kimi-code, glm-code, deepseek-tui —\nanything that reads `.github\u002Fprompts\u002F*.prompt.md`), then **type the\ncommands below into that same coding-agent chat** — not a plain shell.\nThe coding agent handles environment setup for you (`pip install`,\nauth, data download); you never leave the chat.\n\n**1. Optimize a skill on a shipped benchmark** (edit `skill.md` only).\nOpen `copilot_example\u002Flivemath` (or `spreadsheetbench` \u002F `alfworld` \u002F\n`docvqa` \u002F `officeqa` \u002F `searchqa`) with your coding agent, then type\nat the chat prompt:\n\n```\n\u002Fskillopt-loop rounds=10 batch=40 target=gpt-5.4-nano\n```\n\n**2. Co-optimize skill + agent harness** (also edit the Python code).\nOpen `harness_example\u002Fspreadsheetbench` with your coding agent, then\ntype at the chat prompt:\n\n```\n\u002Fharnessopt-loop rounds=2 batch=40 target=gpt-5.4-nano skill=skill_best_nano.md\n```\n\n**3. Bring your own repo + data** — no shim to write; point the coding\nagent at a reference env and let it scaffold. Sample vibe:\n\n> Read `copilot_example\u002Flivemath\u002F` as a reference (env code +\n> `.github\u002Fprompts\u002Fskillopt-loop.prompt.md`). Build a new env at\n> `copilot_example\u002Fmyrepo\u002F` that runs my eval:\n> `python -m myrepo.eval --items X --skill Y --out results.jsonl`.\n> My raw data is a single `data\u002Fall.jsonl` — split it **2:2:6**\n> (train \u002F val \u002F test) with a fixed seed. Each result row has\n> `{id, input, expected, predicted, success}`. Also copy and adapt\n> the slash-command prompt files to my layout — `skillopt-loop.prompt.md`\n> for skill-only, and `harnessopt-loop.prompt.md` (see\n> `harness_example\u002Fspreadsheetbench\u002F.github\u002Fprompts\u002F`) if I want harness\n> co-optimization; rewrite their split sizes, allow-listed editable files,\n> and rollback tag prefix to match my directory. Smoke-test with\n> `--eval_limit 5` and confirm `samples\u002Ffailed\u002F*.md` is non-empty.\n\nOnce it reports done, run flow #1 or #2 against the new folder.\n\nIn all three cases the coding agent drives the loop — rollouts, sample\ninspection, `skill.md` patches, val-gated keep-or-revert, archive to\n`workspace\u002F.skillopt\u002Fhistory\u002F`. You don't touch anything else. When it\nstops improving, `workspace\u002Fskill.md` is the artifact you ship.\n\n**Two slash commands, one loop:**\n\n| Command             | Cwd it expects                              | What the coding agent is allowed to edit                   |\n| ------------------- | ------------------------------------------- | ---------------------------------------------------------- |\n| `\u002Fskillopt-loop`    | inside `copilot_example\u002F\u003Cenv>\u002F`             | just `skill.md`                                            |\n| `\u002Fharnessopt-loop`  | inside `harness_example\u002F\u003Cenv>\u002F`             | `skill.md` **and** the agent code (`rollout.py`, `react_agent.py`, `codegen_agent.py`, `executor.py`, `adapter.py`) |\n\nBoth are just prompt files at `\u003Cenv>\u002F.github\u002Fprompts\u002F*.prompt.md` — read\nthem to tune the loop policy (round count, gate discipline, dead-band,\nroll-back tag names).\n\n## Pipeline\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"images\u002FPipeline.png\" alt=\"SkillOptLite \u002F HarnessOpt pipeline\" width=\"820\"\u002F>\n\u003C\u002Fp>\n\n## Performance\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"images\u002FPerformance.png\" alt=\"SkillOptLite vs SkillOpt across 6 benchmarks\" width=\"900\"\u002F>\n\u003C\u002Fp>\n\nA few concrete headlines from the release ckpts:\n\n| Setup                                    | Score |\n| ---------------------------------------- | ----- |\n| SpreadsheetBench, GPT-5.4-nano + HarnessOpt (`skill_best_nano.md`) | **0.7758** |\n| SpreadsheetBench, GPT-5.5 baseline harness + full SkillOpt          | 0.7620 |\n| LiveMath, GPT-5.5 + SkillOptLite         | **+8.8** pts over SkillOpt |\n| LiveMath, GPT-5.4-nano + SkillOptLite    | **+25.4** pts over SkillOpt |\n| ALFWorld, GPT-5.4-nano + SkillOptLite    | **81.3** (+9.5 over SkillOpt) |\n| Spreadsheet, avg + SkillOptLite          | **+12.6** pts over SkillOpt |\n\nAll numbers come from the ckpts in this repo — reproduce them with\n`scripts\u002Feval_only.py` (see [Direct eval](#run-harnessopt-loop-skill--harness-code) below).\n\n---\n\n## What's in this repo\n\n```\ncopilot_example\u002F                  ← 6 benchmarks, run.sh + \u002Fskillopt-loop each\n├── searchqa\u002F    livemath\u002F    alfworld\u002F\n├── docvqa\u002F      officeqa\u002F    spreadsheetbench\u002F\n├── env.sh                        thin auth delegator\n└── make_samples.py               results.jsonl → per-item .md for the loop\n\nharness_example\u002F                  ← full harness training loop\n└── spreadsheetbench\u002F             baseline code + starting skills + \u002Fharnessopt-loop\n\nskillopt_lite_ckpt\u002F               ← baked skills produced by \u002Fskillopt-loop\n└── \u003Cenv>\u002F\u003Cmodel>\u002Fskill.md        (per env × per target model)\n\nharnessopt_ckpt\u002F                  ← end-state of \u002Fharnessopt-loop\n└── spreadsheetbench\u002F             optimized code + best_skill_*.md snapshot\n\nskillopt_ckpt\u002F                    ← paper GPT-5.5 reference skills\n└── \u003Cenv>\u002Fgpt5.5_skill.md\n\nskillopt\u002F  scripts\u002F  configs\u002F     ← runtime the run.sh scripts call into\ndata\u002F                             ← download.py + local split cache\ndocs\u002F                             ← quickstart, auth, data\nenv.sh  .env.example              ← 3-mode auth bootstrap\n```\n\nDatasets are hosted on 🤗 [`yshenaw\u002FSkillOpt_Lite_Benchmarks`](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fyshenaw\u002FSkillOpt_Lite_Benchmarks).\n\n---\n\n## Setup (fallback — your coding agent normally handles this)\n\nSkip this section unless you're running headless or want to prep the\nenvironment before opening the coding agent. In the normal flow, the\nfirst time you issue any TL;DR command your coding agent will `pip\ninstall -r requirements.txt`, prompt you to fill in `.env` for whichever\nauth mode fits (`az login` \u002F OpenAI key \u002F OpenAI-compatible endpoint),\n`source .\u002Fenv.sh`, and `python data\u002Fdownload.py` before starting the\nloop.\n\nManual bootstrap:\n\n```bash\npython -m venv .venv && source .venv\u002Fbin\u002Factivate\npip install -r requirements.txt\ncp .env.example .env && $EDITOR .env       # AZURE_OPENAI_ENDPOINT \u002F OPENAI_API_KEY \u002F ...\nsource .\u002Fenv.sh\npython data\u002Fdownload.py                     # ~a few hundred MB\n```\n\nFull auth walkthrough (Azure CLI \u002F Azure key \u002F OpenAI \u002F OpenAI-compatible)\nin [docs\u002Fauthentication.md](docs\u002Fauthentication.md).\n\n---\n\n## Roadmap \u002F TODO\n\n- [ ] **Agent-agnostic loop runner.** A drop-in script that wraps `\u002Fskillopt-loop`\n  around *any* existing agent codebase (not just the `copilot_example\u002F` and\n  `harness_example\u002F` layouts). Bring your own `rollout(item) → trace, score`\n  and a `skill.md` entry point, get the same round-based\n  train → patch → val-gate → rollback loop. (Coming soon.)\n- [ ] **Codex CLI plugin.** Port the `\u002Fskillopt-loop` and `\u002Fharnessopt-loop`\n  slash-commands to OpenAI Codex CLI so the same loop works outside VS Code.\n- [ ] **Claude Code plugin.** Same, for Anthropic Claude Code — package the\n  prompts as a slash-command extension.\n\nContributions welcome — open an issue at\n[EvolvingLMMs-Lab\u002FSkillOpt-Lite](https:\u002F\u002Fgithub.com\u002FEvolvingLMMs-Lab\u002FSkillOpt-Lite\u002Fissues)\nif you want to help land any of the above.\n\n---\n\n## Cite\n\n```bibtex\n@article{shen2026skilloptlite,\n  title  = {SkillOpt-Lite: Better and Faster Agent Self-evolution with One Line of Vibe},\n  author = {Shen, Yifei and Li, Bo and Zhang, Xinjie},\n  year   = {2026},\n  note   = {arXiv link coming soon}\n}\n```\n\n---\n\n## Contact\n\n- WeChat community: [scan the QR above](images\u002Fwechat-group.jpg)\n- Correspondence: `yshenaw@connect.ust.hk`\n- Issues \u002F PRs: welcome at [EvolvingLMMs-Lab\u002FSkillOpt-Lite](https:\u002F\u002Fgithub.com\u002FEvolvingLMMs-Lab\u002FSkillOpt-Lite)\n\n## Acknowledgements\n\nBuilt on top of [**SkillOpt**](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FSkillOpt) — the\ntext-space optimizer that trains reusable natural-language skills for frozen\nLLM agents through trajectory-driven edits, validation-gated updates, and\ndeployable `best_skill.md` artifacts. `SkillOpt-Lite` is the minimal\ncoding-agent-driven variant of that loop; `HarnessOpt` extends the same\nloop to also edit the agent code. Huge thanks to the SkillOpt authors and\ncontributors for the original design and open-source release.\n\nReleased under the [MIT License](LICENSE).\n","SkillOpt-Lite 是一个面向大模型智能体（LLM agent）的轻量级自演化优化框架，支持单指令驱动的技能（skill）或技能+执行框架（harness）联合迭代优化。其核心采用闭环提示工程与自动化评估反馈机制，通过预置基准（如 AlfWorld、DocVQA、SpreadsheetBench）和可插拔的 slash-command 接口（如 \u002Fskillopt-loop），实现无需手动环境配置的端到端优化流程。项目强调与主流编码代理（Copilot Chat、Claude Code、Kimi Code 等）深度集成，适配基于 GitHub Actions 风格提示目录（.github\u002Fprompts\u002F）的代理开发范式，适用于智能体能力评测、任务专项调优及低代码代理增强等场景。",2,"2026-07-09 02:30:23","CREATED_QUERY"]