[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-81002":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":13,"subscribersCount":13,"size":13,"stars1d":16,"stars7d":16,"stars30d":16,"stars90d":13,"forks30d":13,"starsTrendScore":17,"compositeScore":13,"rankGlobal":10,"rankLanguage":10,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":10,"pushedAt":10,"updatedAt":28,"readmeContent":29,"aiSummary":30,"trendingCount":13,"starSnapshotCount":13,"syncStatus":15,"lastSyncTime":31,"discoverSource":32},81002,"PureDocBench","zhihengli-casia\u002FPureDocBench","zhihengli-casia","PureDocBench: source-traceable benchmark for document parsing across clean, degraded, and real-world settings","https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fzhihengli-casia\u002Fpuredocbench",null,"Python",31,0,30,2,1,3,"Other",false,"main",true,[23,24,25,26,27],"benchmark","document-ai","document-parsing","ocr","synthetic-data","2026-06-12 02:04:09","# PureDocBench\n\n\u003Cp align=\"center\">\n  \u003Cstrong>How far is document parsing from solved?\u003C\u002Fstrong>\u003Cbr>\n  A source-traceable benchmark for OCR and document parsing across clean, digitally degraded, and real-degraded document settings.\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fzhihengli-casia\u002Fpuredocbench\">\u003Cimg alt=\"Hugging Face Dataset\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDataset-Hugging%20Face-yellow\">\u003C\u002Fa>\n  \u003Ca href=\"LICENSE_DATA\">\u003Cimg alt=\"Data License\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FData-CC%20BY%204.0-lightgrey\">\u003C\u002Fa>\n  \u003Ca href=\"LICENSE\">\u003Cimg alt=\"Code License\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FCode-MIT-green\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.07492\">\u003Cimg alt=\"arXiv\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2605.07492-b31b1b\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"docs\u002FREADME_ZH.md\">Chinese README\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fzhihengli-casia\u002Fpuredocbench\">Dataset\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.07492\">Paper\u003C\u002Fa>\n\u003C\u002Fp>\n\nPureDocBench uses HTML\u002FCSS document sources as hidden anchors: each page is rendered into images and annotated from the same structured source. This gives a benchmark where text, tables, formulas, captions, and reading order can be scored with less post-hoc annotation noise.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig3_data_overview_final.png\" alt=\"PureDocBench overview\" width=\"92%\">\n\u003C\u002Fp>\n\n## At A Glance\n\n| Item | Count |\n|---|---:|\n| Official pages | 1,475 |\n| Official images | 4,425 |\n| Top-level domains | 10 |\n| Fine-grained subcategories | 66 |\n| Image tracks | clean, digital-degraded, real-degraded |\n| Scored structures | text, formulas, tables, reading order |\n\n## Main Leaderboard\n\nThe paper evaluates 40 systems across pipeline specialists, end-to-end document parsers, and general-purpose VLMs. Table 2 is the main leaderboard: each track reports Overall, TextEdit, FormulaCDM, TableTEDS, and ROEdit; Avg3 averages the three track Overall scores.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ftable_main_leaderboard.png\" alt=\"Table 2: three-track leaderboard on PureDocBench\" width=\"98%\">\n\u003C\u002Fp>\n\n## Diagnostics\n\nThe diagnostic panel shows where current systems still have headroom. Formula recognition is the largest single bottleneck, and real degradation changes rankings more sharply than digital degradation.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig_diagnostic_panels.png\" alt=\"Diagnostic panels\" width=\"96%\">\n\u003C\u002Fp>\n\n## Case Studies\n\nThe four case studies below are all taken from the paper. They show failures that aggregate scores can hide: notation loss, reading-order mistakes, annotation contamination, table-structure errors, character-level corruption, and missing visual authentication cues.\n\n### Case 1: Academic\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig_case_study_academic.png\" alt=\"Case study 1: academic structured lab report\" width=\"96%\">\n\u003C\u002Fp>\n\n### Case 2: Business\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig_case_study_business.png\" alt=\"Case study 2: business product specification table\" width=\"96%\">\n\u003C\u002Fp>\n\n### Case 3: Finance\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig_case_study_actuarial.png\" alt=\"Case study 3: finance actuarial valuation report\" width=\"96%\">\n\u003C\u002Fp>\n\n### Case 4: Certificate\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig_case_study_certificate.png\" alt=\"Case study 4: Chinese product quality certificate\" width=\"96%\">\n\u003C\u002Fp>\n\n## Appendix Highlights\n\nThe appendix documents the degradation design and per-category behavior used to make the benchmark reproducible.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig_degradation_ops.png\" alt=\"Degradation operations\" width=\"96%\">\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig_degradation_scenarios.png\" alt=\"Degradation scenarios\" width=\"96%\">\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigures\u002Ffig_per_category_overview.png\" alt=\"Per-category overview\" width=\"92%\">\n\u003C\u002Fp>\n\n## Download\n\nThe full image\u002FGT\u002FHTML release is hosted on Hugging Face:\n\n```bash\n# After downloading all files from Hugging Face:\nshasum -a 256 -c SHA256SUMS.txt\ncat pdb_full.tar.part-* | tar -xf -\n```\n\nVerify the split archive and reconstructed release:\n\n```bash\npython scripts\u002Fverify_split_archive.py \u002Fpath\u002Fto\u002Fdownloaded\u002Ffiles\n\npython scripts\u002Fvalidate_release_manifest.py \\\n  --release-root \u002Fpath\u002Fto\u002Fpuredocbench-v1.0 \\\n  --manifest manifests\u002Frelease_manifest_candidate_1475.csv\n```\n\n## Inference And Scoring\n\nPureDocBench includes a public CLI for model-agnostic inference, fast lightweight scoring, and OmniDocBench-aligned evaluation. Use `puredocbench score` for quick sanity checks; use `puredocbench score-omnidocbench` with an OmniDocBench checkout for platform-aligned CDM\u002FTEDS numbers.\n\n```bash\npip install -e .\n\npuredocbench infer \\\n  --images \u002Fpath\u002Fto\u002Fpuredocbench-v1.0\u002Fimages\u002Fclean \\\n  --output-dir predictions\u002Fmy_model_clean \\\n  --command-template 'python my_model_infer.py --image {image} --out {output}'\n\npuredocbench score \\\n  --release-root \u002Fpath\u002Fto\u002Fpuredocbench-v1.0 \\\n  --manifest manifests\u002Frelease_manifest_candidate_1475.csv \\\n  --pred-dir predictions\u002Fmy_model_clean \\\n  --track clean \\\n  --limit 20 \\\n  --out-dir scores\u002Fmy_model_clean\n\npuredocbench score-omnidocbench \\\n  --release-root \u002Fpath\u002Fto\u002Fpuredocbench-v1.0 \\\n  --manifest manifests\u002Frelease_manifest_candidate_1475.csv \\\n  --pred-dir predictions\u002Fmy_model_clean \\\n  --track clean \\\n  --omnidocbench-root \u002Fpath\u002Fto\u002FOmniDocBench \\\n  --out-dir omnidocbench_scores\u002Fmy_model_clean\n```\n\nSee [docs\u002FINFERENCE_SCORING.md](docs\u002FINFERENCE_SCORING.md) for the full interface and evaluator-version notes.\n\n## Repository Contents\n\n```text\nmanifests\u002F                         Release and sample manifests\nmetadata\u002F                          Dataset card and Croissant metadata\nscripts\u002F                           Rendering, degradation, validation, leaderboard tools\npuredocbench\u002F                      Public inference, scoring, and OmniDocBench export CLI\nmodel_inference\u002F                   Sanitized model inference configs and runners\nsupplemental_inference_scoring\u002F    API\u002Flocal inference and scoring utilities\nassets\u002Ffigures\u002F                    Figures from the paper\n```\n\n## Quick Start\n\n```bash\npython -m venv .venv\nsource .venv\u002Fbin\u002Factivate\npip install -r requirements.txt\nplaywright install chromium\n```\n\nRender one HTML page:\n\n```bash\npython scripts\u002Frender_single_image.py \\\n  --html \u002Fpath\u002Fto\u002Fpage.html \\\n  --out \u002Fpath\u002Fto\u002Fpage.png \\\n  --dpi 300\n```\n\nApply a deterministic degradation profile:\n\n```bash\npython scripts\u002Fapply_degradation_ablation.py \\\n  --input \u002Fpath\u002Fto\u002Fclean_images \\\n  --output \u002Fpath\u002Fto\u002Fdegraded_images \\\n  --profile full_medium\n```\n\n## License\n\n- Dataset assets are released under **CC BY 4.0**; see [LICENSE_DATA](LICENSE_DATA).\n- Code in this repository is released under the license in [LICENSE](LICENSE).\n- Model weights are not redistributed.\n\n## Citation\n\n```bibtex\n@article{li2026puredocbench,\n  title   = {How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings},\n  author  = {Li, Zhiheng and Ma, Zongyang and Chen, Jiaxian and Zhang, Jianing and Su, Zhaolong and Zhang, Yutong and Yu, Zhiyin and Liu, Ruiqi and Lv, Xiaolei and Li, Bo and Gao, Jun and Zhang, Ziqi and Yuan, Chunfeng and Li, Bing and Hu, Weiming},\n  journal = {arXiv preprint arXiv:2605.07492},\n  year    = {2026},\n  doi     = {10.48550\u002FarXiv.2605.07492},\n  url     = {https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.07492}\n}\n```\n","PureDocBench 是一个用于评估文档解析和OCR性能的基准测试工具，涵盖了从干净、数字降级到真实世界文档的各种场景。它通过将HTML\u002FCSS源文件渲染成图像并基于同一结构化源进行标注，实现了对文本、表格、公式、标题及阅读顺序等元素的准确评分，减少了事后标注带来的噪声。该项目特别适合于需要在不同质量水平下测试文档解析算法有效性的研究者和开发者使用。此外，PureDocBench还提供了详细的诊断面板帮助识别现有系统中的改进空间，并支持跨多种类型的文档（如学术、商业）进行案例分析。","2026-06-11 04:03:08","CREATED_QUERY"]