[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-93346":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":8,"htmlUrl":8,"language":9,"languages":8,"totalLinesOfCode":8,"stars":10,"forks":11,"watchers":12,"openIssues":13,"contributorsCount":13,"subscribersCount":13,"size":13,"stars1d":13,"stars7d":14,"stars30d":14,"stars90d":13,"forks30d":13,"starsTrendScore":15,"compositeScore":16,"rankGlobal":8,"rankLanguage":8,"license":17,"archived":18,"fork":18,"defaultBranch":19,"hasWiki":20,"hasPages":18,"topics":21,"createdAt":8,"pushedAt":8,"updatedAt":22,"readmeContent":23,"aiSummary":24,"trendingCount":13,"starSnapshotCount":13,"syncStatus":25,"lastSyncTime":26,"discoverSource":27},93346,"wandr","perplexityai\u002Fwandr","perplexityai",null,"Python",228,24,1,0,84,41,69.59,"Apache License 2.0",false,"main",true,[],"2026-07-22 04:02:08","# WANDR\n\nWANDR is a benchmark for wide and deep research: structured, high-volume information\nwork that requires broad discovery, extensive enrichment, systematic extraction,\nprecise entity disambiguation, and evidence-backed answer synthesis.\n\nSee the technical report in [references](reference\u002Fwandr_paper.pdf).\n\nCurrent implementation is split into four independent layers:\n\n1. **WANDR source data.** `reference\u002Fwandr_tasks\u002F` contains the editable task\n   configs, schemas, prompts, and artifacts in WANDR's source format. The source\n   tree is portable data and can be hosted independently of this repository.\n2. **WANDR-to-Harbor adapter.** `adapters\u002Fwandr\u002F` consumes a WANDR source tree\n   and deterministically produces Harbor task packages. It owns format\n   translation and packaging, not benchmark semantics.\n3. **Harbor-format WANDR tasks.** `datasets\u002Fwandr\u002F` contains the generated,\n   self-contained task packages that are also published through the Harbor\n   registry. Each package carries its instruction, environment, evaluator, and\n   task-specific scoring material.\n4. **Relay.** `agents\u002Frelay\u002F` is a generic Harbor agent for repository-patch and\n   file-output benchmarks. It adapts standard request\u002Fcompletion-style remote\n   endpoints to Harbor by collecting their output files and materializing them\n   in the task workspace; it is not WANDR-specific.\n\n## Quick Start\n\n### Requirements\n\n- Python 3.12\n- [uv](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002F)\n- Docker with a running daemon for local task environments\n- API keys for the providers used by the selected config\n\nClone the repository and install the locked workspace:\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fppl-ai\u002Fwandr.git\ncd wandr\nuv --no-config sync --locked\n```\n\nCreate a local environment file:\n\n```bash\ncp .env.example .env\n```\n\nSet the keys you intend to use, then run the repository checks:\n\n```bash\n.\u002Fscripts\u002Fwandr check\n```\n\nThe cheapest end-to-end run uses the `smoke` task and one low-effort OpenAI\nsolver. It still makes paid OpenAI and Perplexity API calls:\n\n```bash\n.\u002Fscripts\u002Fwandr smoke-local\n```\n\nThe first local run builds the task image and can take longer than subsequent\nruns. Harbor writes run state and results under `jobs\u002F`.\n\n### E2B\n\nE2B is optional and may add execution charges. Set `E2B_API_KEY` in `.env`, then\nrun the same smoke workflow in an E2B environment:\n\n```bash\n.\u002Fscripts\u002Fwandr smoke-e2b\n```\n\nThe E2B path builds from the checked-in task Dockerfile. It uses the same task,\nRelay, and verifier contracts as the local Docker path.\n\n## Credentials\n\n`.env` is ignored by Git. The wrapper parses it as dotenv data before invoking\nHarbor; it never executes the file as shell code. Exported environment variables\ntake precedence over values in `.env`.\n\n| Variable | Used for |\n| --- | --- |\n| `OPENAI_API_KEY` | OpenAI Relay runs and WANDR judge calls |\n| `ANTHROPIC_API_KEY` | Anthropic Managed Agents Relay runs |\n| `PERPLEXITY_API_KEY` | Perplexity Relay runs and verifier page fetching |\n| `EXA_API_KEY` | Exa Agent Relay runs |\n| `PARALLEL_API_KEY` | Parallel Task API Relay runs |\n| `GEMINI_API_KEY` or `GOOGLE_API_KEY` | Gemini Deep Research Relay runs |\n| `E2B_API_KEY` | Optional E2B execution environment |\n\n`configs\u002Fsmoke.yaml` needs OpenAI and Perplexity keys. The all-provider configs\nneed all six provider keys. E2B runs additionally need `E2B_API_KEY`.\n\n## Run Configs\n\nThe checked-in configs form an increasing cost and coverage ladder:\n\n| Config | Coverage | Purpose |\n| --- | --- | --- |\n| `configs\u002Fsmoke.yaml` | One smoke task, one low-effort endpoint | Fast local end-to-end check |\n| `configs\u002Fsmoke-all.yaml` | One smoke task, all Relay endpoints at low\u002Ffast settings | Provider integration check |\n| `configs\u002Fvalidation.yaml` | Two representative tasks across all endpoints | Standard release validation |\n| `configs\u002Fwandr.yaml` | Full scored task set across all endpoints | Full benchmark run |\n\nRun the all-provider smoke config directly through the wrapper:\n\n```bash\n.\u002Fscripts\u002Fwandr run -y -c configs\u002Fsmoke-all.yaml\n```\n\nRun the standard two-task validation:\n\n```bash\n.\u002Fscripts\u002Fwandr validate\n```\n\nRun the full benchmark only after smoke and validation have passed:\n\n```bash\n.\u002Fscripts\u002Fwandr run -y -c configs\u002Fwandr.yaml\n```\n\nThese commands make paid calls to solver, fetch, and judge APIs. The validation\nand full configs fan out across six providers; the full config applies that\nmatrix to the entire dataset and can be very expensive. Check each provider's\ncurrent pricing and account limits before starting them. Harbor does not impose\na spending cap.\n\nUse the generic wrapper for custom Harbor arguments or edited configs:\n\n```bash\n.\u002Fscripts\u002Fwandr run \u003Charbor-run-arguments>\n```\n\n## How A Run Works\n\n```text\nHarbor task environment\n  -> Relay snapshots the workspace and invokes one configured endpoint\n  -> Relay materializes the endpoint's declared files in \u002Fworkspace\n  -> Harbor starts the task-local WANDR verifier\n  -> WANDR fetches submitted pages, normalizes entities, deduplicates, and judges\n  -> Harbor records rewards, diagnostics, reports, and Relay observability\n```\n\nEach generated task is independently runnable and contains:\n\n- `instruction.md`: solver-facing task instructions;\n- `task.toml`: Harbor metadata, resources, verifier environment, and the\n  task-owned `metadata.required_file_paths` output contract;\n- `environment\u002F`: the public Docker build;\n- `tests\u002Fwandr_task\u002F`: task-specific configuration, schemas, prompt fragments,\n  and required artifacts;\n- `tests\u002Fwandr_core\u002F`: a vendored copy of the evaluator runtime;\n- `tests\u002Fmanifest.json`: the ordered WANDR task names consumed by the verifier.\n\nThe task-local evaluator is deliberate. A task can be published, downloaded,\nand verified without requiring a separate WANDR package at runtime.\nGenerated tasks do not ship example solver outputs or a bundled solver;\nthe verifier always grades the required files already present in `\u002Fworkspace`.\nTask-local tests may contain compact grader-only rubrics and task-owned public\nevidence assets. Neither is copied into the agent workspace or rendered into\nthe solver instruction unless it is explicitly part of the task input.\n\n## Outputs\n\nHarbor creates `jobs\u002F\u003Crun-id>\u002F` with a directory per trial. The most useful\nfiles are:\n\n- `\u003Ctrial>\u002Fresult.json`: Harbor's trial result;\n- `\u003Ctrial>\u002Fagent\u002Fevents.jsonl`: provider-neutral Relay events;\n- `\u003Ctrial>\u002Fagent\u002Ftrajectory.json`: the ATIF trajectory;\n- `\u003Ctrial>\u002Fagent\u002Fstatus.json`: Relay lifecycle status;\n- `\u003Ctrial>\u002Fverifier\u002Freward.json`: primary and named rewards;\n- `\u003Ctrial>\u002Fverifier\u002Fwandr_metrics.json`: WANDR metric rollups;\n- `\u003Ctrial>\u002Fverifier\u002Freport.html`: detailed human-readable report;\n- `\u003Ctrial>\u002Fverifier\u002Fwandr-details.json`: verifier diagnostics;\n- `\u003Ctrial>\u002Fverifier\u002Ferror.json`: setup or evaluator failure, when present.\n\nA completed zero reward is a scored result. An `error.json` means the verifier\ndid not produce a valid score.\n\n## Task Development\n\nThe files under `datasets\u002Fwandr\u002F` are generated. Make task edits in\n`reference\u002Fwandr_tasks\u002F`, shared evaluator edits in\n`adapters\u002Fwandr\u002Fsrc\u002Fwandr\u002Forigin\u002Fwandr_core\u002F`, and generic Harbor wrapper edits\nin `adapters\u002Fwandr\u002Fsrc\u002Fwandr\u002Ftask-template\u002F`.\n\nRegenerate one task by its underscore-delimited WANDR name:\n\n```bash\nuv --no-config run --project adapters\u002Fwandr --locked wandr \\\n  pharma_former_rd_heads --overwrite\n```\n\nRegenerate every task:\n\n```bash\nuv --no-config run --project adapters\u002Fwandr --locked wandr --overwrite\n```\n\nGeneration also refreshes `datasets\u002Fwandr\u002Fdataset.toml`. Do not hand-edit a\ngenerated task to diverge from its source or the adapter templates. The\nrepository check verifies task sources, vendored evaluator copies, wrapper files, and\ndataset digests:\n\n```bash\n.\u002Fscripts\u002Fwandr check\n```\n\nAdapter details live in [`adapters\u002Fwandr\u002FREADME.md`](adapters\u002Fwandr\u002FREADME.md).\nRelay details live in [`agents\u002Frelay\u002FREADME.md`](agents\u002Frelay\u002FREADME.md).\nTask-source details live in\n[`reference\u002Fwandr_tasks\u002FREADME.md`](reference\u002Fwandr_tasks\u002FREADME.md).\n\n## Data And Third-Party Sources\n\nRequired task artifacts can include derived public-record material. Third-party\nsource material remains subject to its own terms; follow linked source terms\nwhen reusing or redistributing it.\n\n## Troubleshooting\n\n- `uv lock --check` failures: run `uv --no-config sync --locked` from the repository root\n  and confirm that the checked-in lockfile has not been changed locally.\n- Docker connection failures: start Docker and verify `docker info` succeeds.\n- Missing credentials: compare `.env` with `.env.example`; the preflight check\n  reports the required variables for the selected workflow.\n- Provider failures: inspect `\u003Ctrial>\u002Fagent\u002Fstatus.json`, `events.jsonl`, and\n  `result.json` before retrying.\n- Verifier failures: inspect `\u003Ctrial>\u002Fverifier\u002Ferror.json`, `setup.log`, and\n  `wandr\u002Fstdio.log`. Missing reward files are verifier failures, not zero scores.\n- Slow first run: allow time for the Docker or E2B image build and locked Python\n  dependency installation. Later runs reuse those caches.\n","WANDR 是一个面向宽域深度研究任务的基准测试框架，用于评估AI系统在大规模结构化信息处理中的综合能力。其核心功能涵盖广度发现、信息富化、系统化抽取、实体消歧及证据驱动的答案合成；技术上采用分层架构（源数据、适配器、Harbor任务包、通用中继代理），支持本地Docker或E2B云环境执行，并依赖多API提供商（如OpenAI、Perplexity、Exa）完成端到端评测。适用于评估复杂推理、多跳检索、事实验证与研究型问答等高阶AI能力场景。",2,"2026-07-17 02:30:03","CREATED_QUERY"]