[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-80850":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":12,"stars7d":15,"stars30d":16,"stars90d":14,"forks30d":14,"starsTrendScore":15,"compositeScore":17,"rankGlobal":9,"rankLanguage":9,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":9,"pushedAt":9,"updatedAt":23,"readmeContent":24,"aiSummary":25,"trendingCount":14,"starSnapshotCount":14,"syncStatus":12,"lastSyncTime":26,"discoverSource":27},80850,"cloakFetch","Agents365-ai\u002FcloakFetch","Agents365-ai","Claude Code PostToolUse hook: when WebFetch hits Cloudflare\u002F403, transparently retry through CloakBrowser headlessly and inject clean markdown via defuddle.",null,"Python",43,2,36,0,6,7,49.13,"MIT License",false,"main",true,[],"2026-06-12 04:01:30","# cloakFetch — pass Cloudflare-protected URLs through Claude Code 🛡️\n\n[![License: MIT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-yellow.svg)](LICENSE)\n[![GitHub stars](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002FAgents365-ai\u002FcloakFetch?style=flat&logo=github)](https:\u002F\u002Fgithub.com\u002FAgents365-ai\u002FcloakFetch\u002Fstargazers)\n[![GitHub forks](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fforks\u002FAgents365-ai\u002FcloakFetch?style=flat&logo=github)](https:\u002F\u002Fgithub.com\u002FAgents365-ai\u002FcloakFetch\u002Fnetwork\u002Fmembers)\n[![Latest Release](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fv\u002Frelease\u002FAgents365-ai\u002FcloakFetch?logo=github)](https:\u002F\u002Fgithub.com\u002FAgents365-ai\u002FcloakFetch\u002Freleases\u002Flatest)\n[![Last Commit](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flast-commit\u002FAgents365-ai\u002FcloakFetch?logo=github)](https:\u002F\u002Fgithub.com\u002FAgents365-ai\u002FcloakFetch\u002Fcommits\u002Fmain)\n\n[![Claude Code Hook](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FClaude%20Code-PostToolUse%20hook-8a2be2)](https:\u002F\u002Fdocs.claude.com\u002Fen\u002Fdocs\u002Fclaude-code\u002Fhooks)\n[![Agent Skills](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FAgent%20Skills-compatible-2ea44f)](https:\u002F\u002Fagentskills.io)\n[![Discord](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDiscord-Join-5865F2?logo=discord&logoColor=white)](https:\u002F\u002Fdiscord.gg\u002F79JF5Atuk)\n\n**English** · [中文](README_CN.md)\n\nExternal references: [CloakBrowser](https:\u002F\u002Fgithub.com\u002FCloakHQ\u002FCloakBrowser) · [trafilatura](https:\u002F\u002Fgithub.com\u002Fadbar\u002Ftrafilatura) · [Claude Code hooks](https:\u002F\u002Fdocs.claude.com\u002Fen\u002Fdocs\u002Fclaude-code\u002Fhooks)\n\nTwo paths for the same idea: when a web fetch is blocked by [Cloudflare](https:\u002F\u002Fwww.cloudflare.com\u002F) (or similar bot protection), route the URL through [CloakBrowser](https:\u002F\u002Fgithub.com\u002FCloakHQ\u002FCloakBrowser) — a stealth Chromium that passes the JS challenge — and return clean markdown via [trafilatura](https:\u002F\u002Fgithub.com\u002Fadbar\u002Ftrafilatura).\n\n- **[Path A: PostToolUse hook](#path-a--claude-code-posttooluse-hook)** — fully automatic on Claude Code. Every blocked `WebFetch` silently falls back; the agent never sees the failure.\n- **[Path B: SKILL.md skill](#path-b--skillmd-skill-for-hookless-agents)** — reactive fallback for any SKILL.md-aware agent (Codex, OpenCode, OpenClaw, SkillsMP). Agent decides to invoke it after seeing a 403\u002FCF pattern.\n\n## Why\n\nClaude Code's built-in `WebFetch` (and `curl`, and `requests`, and most HTTP clients) can't pass Cloudflare's JS challenge — so any request to a CF-protected site (`science.org`, many publishers, lots of news sites) comes back as:\n\n```\nThe server returned HTTP 403 Forbidden.\n```\n\nCloakBrowser is a real Chromium with anti-bot patches at the C++ level that *does* pass those challenges. cloakFetch wires CloakBrowser + trafilatura into Claude Code (and other agents) so the agent never has to tell the user \"this page is unfetchable.\"\n\n## Two activation paths\n\n|  | Path A: Hook | Path B: Skill |\n|---|---|---|\n| **Trigger** | Automatic — fires on every WebFetch result | Reactive — agent decides after seeing a failed fetch |\n| **Agent cognition** | Zero — invisible upgrade | Has to notice 403\u002FCF pattern + recall skill |\n| **Runtime support** | Claude Code only (needs `PostToolUse` hook system) | Any SKILL.md-aware agent: Claude Code, OpenClaw, Codex, OpenCode, SkillsMP |\n| **Latency on hit** | ~25–40 s | ~25–40 s |\n| **Latency on miss** | ~milliseconds (regex check, no browser) | None (skill not invoked) |\n| **Install** | Copy 2 scripts + edit `~\u002F.claude\u002Fsettings.json` | Drop skill folder into the agent's skills dir |\n| **Files** | `hooks\u002Fcloak_fetch.py` + `hooks\u002Fwebfetch_cloak_fallback.sh` | `skills\u002Fcloak-fetch\u002FSKILL.md` + `cloak_fetch.py` + `cloak_fetch.sh` |\n\nSame `cloak_fetch.py` underneath both — the difference is just *how* it gets activated.\n\n## Repo layout\n\n```\ncloakFetch\u002F\n├── hooks\u002F                          # Path A — Claude Code PostToolUse\n│   ├── cloak_fetch.py              #   headless CloakBrowser → rendered HTML\n│   └── webfetch_cloak_fallback.sh  #   payload matcher + orchestrator\n├── skills\u002Fcloak-fetch\u002F             # Path B — SKILL.md skill\n│   ├── SKILL.md                    #   pushy description + trigger heuristics\n│   ├── cloak_fetch.py              #   (same script, env-python shebang)\n│   └── cloak_fetch.sh              #   wrapper: locate python, fetch, extract\n├── settings.snippet.json           #   PostToolUse JSON block to paste into ~\u002F.claude\u002Fsettings.json\n└── README.md\n```\n\n---\n\n## Path A — Claude Code PostToolUse hook\n\n### Architecture\n\n```\n┌───────────────────┐    fails (CF 403)\n│  WebFetch (built- │ ─────────────────┐\n│  in Claude tool)  │                  │\n└───────────────────┘                  ▼\n                          ┌──────────────────────────────┐\n                          │ webfetch_cloak_fallback.sh   │\n                          │ (PostToolUse hook)           │\n                          │                              │\n                          │ 1. read payload from stdin   │\n                          │ 2. regex-match failure       │\n                          │ 3. extract tool_input.url    │\n                          │ 4. call cloak_fetch.py       │\n                          │ 5. emit additionalContext    │\n                          └──────────────────────────────┘\n                                         │\n                                         ▼\n                          ┌──────────────────────────────┐\n                          │ cloak_fetch.py               │\n                          │ (CloakBrowser headless)      │\n                          │                              │\n                          │ launch → goto → wait for CF  │\n                          │ to clear → wait for content  │\n                          │ → trafilatura → markdown to  │\n                          │ stdout                       │\n                          └──────────────────────────────┘\n```\n\nTwo independent files so failure-detection regex (bash) and browser logic (Python) evolve separately.\n\n### Install (hook)\n\n```bash\n# 1. Copy the hook scripts into Claude Code's hook directory\nmkdir -p ~\u002F.claude\u002Fhooks\ncp hooks\u002Fcloak_fetch.py hooks\u002Fwebfetch_cloak_fallback.sh ~\u002F.claude\u002Fhooks\u002F\nchmod +x ~\u002F.claude\u002Fhooks\u002Fcloak_fetch.py ~\u002F.claude\u002Fhooks\u002Fwebfetch_cloak_fallback.sh\n\n# 2. Tell the hook where your CloakBrowser-enabled Python lives. Add to your\n#    shell rc (~\u002F.zshrc, ~\u002F.bashrc, etc.):\n#      export CLOAKBROWSER_PYTHON=\"$HOME\u002Fpath\u002Fto\u002FCloakBrowser\u002F.venv\u002Fbin\u002Fpython\"\n#    The hook also auto-tries $HOME\u002Fgithub\u002FCloakBrowser\u002F.venv\u002Fbin\u002Fpython and\n#    `python3` as fallbacks, so you can skip this if either of those works.\n\n# 3. Register the hook in ~\u002F.claude\u002Fsettings.json — add the contents of\n#    settings.snippet.json as a new entry inside the \"PostToolUse\" array.\n#    Example final shape:\n```\n\n```json\n{\n  \"hooks\": {\n    \"PostToolUse\": [\n      {\n        \"matcher\": \"WebFetch\",\n        \"hooks\": [\n          {\n            \"type\": \"command\",\n            \"command\": \"$HOME\u002F.claude\u002Fhooks\u002Fwebfetch_cloak_fallback.sh\"\n          }\n        ]\n      }\n    ]\n  }\n}\n```\n\nThe hook becomes active on the next tool call — no Claude Code restart required.\n\n#### Tip — symlink instead of copy\n\nIf you keep this repo as a git checkout (recommended), symlink the hook files into `~\u002F.claude\u002Fhooks\u002F` instead of copying them. Edits in the repo are then live immediately, and `git pull` updates your hooks:\n\n```bash\nmkdir -p ~\u002F.claude\u002Fhooks\nln -sf \"$(pwd)\u002Fhooks\u002Fcloak_fetch.py\"             ~\u002F.claude\u002Fhooks\u002Fcloak_fetch.py\nln -sf \"$(pwd)\u002Fhooks\u002Fwebfetch_cloak_fallback.sh\" ~\u002F.claude\u002Fhooks\u002Fwebfetch_cloak_fallback.sh\n```\n\nTradeoff: if you move or delete the repo checkout, the hook silently stops firing (the script's `[ ! -f \"$CLOAK_FETCH\" ]` check trips and exits 0). For a stable checkout under `~\u002Fgithub\u002F`, that's a fine bet.\n\n### Test (hook)\n\nSimulate the harness by piping a fake failed-WebFetch payload to the hook:\n\n```bash\necho '{\n  \"tool_name\": \"WebFetch\",\n  \"tool_input\": {\"url\": \"https:\u002F\u002Fwww.science.org\u002Fcontent\u002Fpage\u002Finformation-authors-research-articles\", \"prompt\": \"x\"},\n  \"tool_response\": \"The server returned HTTP 403 Forbidden.\"\n}' | ~\u002F.claude\u002Fhooks\u002Fwebfetch_cloak_fallback.sh\n```\n\nExpected: `{\"hookSpecificOutput\": {\"hookEventName\": \"PostToolUse\", \"additionalContext\": \"WebFetch was blocked... \u003Cmarkdown>\"}}` on stdout, exit code 0.\n\nFor a live test, ask Claude inside a Claude Code session to fetch any Cloudflare-protected URL. You should see the `WebFetch` 403 immediately followed by a `PostToolUse:WebFetch hook additional context: ...` block in the conversation.\n\n### Configuration knobs (hook)\n\nInside `hooks\u002Fwebfetch_cloak_fallback.sh`:\n\n| Variable | Default | Purpose |\n|---|---|---|\n| `CLOAK_FETCH` | `$HOME\u002F.claude\u002Fhooks\u002Fcloak_fetch.py` | Path to the Python fetcher. Override with an env var of the same name. |\n| `CLOAKBROWSER_PYTHON` | `$HOME\u002Fgithub\u002FCloakBrowser\u002F.venv\u002Fbin\u002Fpython` → `python3` | Python interpreter that runs the fetcher. Must have `cloakbrowser` importable. |\n| `FAILURE_REGEX` | `403\\|429\\|forbidden\\|cloudflare\\|just a moment\\|enable javascript and cookies\\|resource was not loaded\\|access denied\\|blocked\\|datadome\\|akamai\\|please verify you are a human\\|incapsula\\|pardon our interruption\\|kasada\\|aws-waf\\|sucuri` | Case-insensitive regex against `tool_response`. Covers Cloudflare + the major bot-protection vendors. Widen \u002F narrow to taste. |\n\n---\n\n## Path B — SKILL.md skill (for hookless agents)\n\nThe hook approach only works on Claude Code. For agents that don't have a `PostToolUse` system — Codex CLI, OpenCode, OpenClaw, SkillsMP — install cloakFetch as a SKILL.md-format skill. The agent reads the SKILL.md when relevant and invokes the wrapper script after recognising a Cloudflare failure pattern.\n\n### Install (skill)\n\n| Agent | Install path |\n|---|---|\n| **Claude Code** (global) | `cp -r skills\u002Fcloak-fetch ~\u002F.claude\u002Fskills\u002Fcloak-fetch` |\n| **Claude Code** (project) | `cp -r skills\u002Fcloak-fetch .claude\u002Fskills\u002Fcloak-fetch` |\n| **OpenClaw** (global) | `cp -r skills\u002Fcloak-fetch ~\u002F.openclaw\u002Fskills\u002Fcloak-fetch` |\n| **OpenClaw** (project) | `cp -r skills\u002Fcloak-fetch skills\u002Fcloak-fetch` |\n| **SkillsMP** | search for `cloak-fetch` on [skillsmp.com](https:\u002F\u002Fskillsmp.com) |\n\nIf your CloakBrowser venv isn't at the default path, set the env var (in your shell rc or per-invocation):\n\n```bash\nexport CLOAKBROWSER_PYTHON=\u002Fpath\u002Fto\u002Fyour\u002Fcloakbrowser\u002F.venv\u002Fbin\u002Fpython\n```\n\n### Invoke (skill)\n\nThe agent runs this single command after a normal fetcher returns a 403\u002FCF pattern:\n\n```bash\n~\u002F.claude\u002Fskills\u002Fcloak-fetch\u002Fcloak_fetch.sh \"\u003CURL>\"\n```\n\nThe wrapper handles everything: finds a `cloakbrowser`-importable Python, launches the headless browser, runs trafilatura, prints clean markdown on stdout. Stderr carries progress messages; exit non-zero on any failure.\n\n### Test (skill)\n\n```bash\n~\u002F.claude\u002Fskills\u002Fcloak-fetch\u002Fcloak_fetch.sh \"https:\u002F\u002Fwww.science.org\u002Fcontent\u002Fpage\u002Finformation-authors-research-articles\"\n```\n\nExpected: ~20–40 s, then ~25 KB of clean markdown on stdout (page title is \"Information for Authors-Research Articles\").\n\nFor a sanity check on a non-Cloudflare site:\n\n```bash\n~\u002F.claude\u002Fskills\u002Fcloak-fetch\u002Fcloak_fetch.sh \"https:\u002F\u002Fexample.com\"\n# → \"This domain is for use in documentation examples...\"\n```\n\n### Configuration knobs (skill)\n\n| Env var | Default | Purpose |\n|---|---|---|\n| `CLOAKBROWSER_PYTHON` | (auto-detect: `~\u002Fgithub\u002FCloakBrowser\u002F.venv\u002Fbin\u002Fpython`, then `python3`) | Python interpreter with `cloakbrowser` importable |\n\nInside `skills\u002Fcloak-fetch\u002Fcloak_fetch.py`:\n\n| Knob | Default | Purpose |\n|---|---|---|\n| `headless=True` | `True` | Flip to `False` to see the browser window for debugging |\n| Selector wait list | `main, article, .article__body, .core-container, .pb-page-body` | Selectors that signal SPA content has rendered. Extend if a target site needs something more specific. |\n| `time.sleep(2)` settle | 2 s | Extra wait for late-loading JS. |\n\n---\n\n## Prerequisites (both paths)\n\n- [CloakBrowser](https:\u002F\u002Fgithub.com\u002FCloakHQ\u002FCloakBrowser) installed with the `cloakbrowser` Python package importable (a venv with `pip install cloakbrowser` works)\n- `trafilatura` installed into the same Python env (`pip install trafilatura`) — used for HTML → markdown extraction\n- Path A only: `jq` (for parsing the hook payload)\n\n## Behaviour & safety\n\n- **Fail-closed.** Both paths leave the original failure intact if something inside cloakFetch breaks (no Python with cloakbrowser, network down, CloakBrowser can't pass the challenge). The agent is never tricked into thinking a fetch succeeded when it didn't.\n- **Silent on the happy path.** The hook does nothing when the regex doesn't match; the skill is simply not invoked when there's no failure to recover from.\n- **Cost.** A triggered fallback runs a real browser — ~20–40 s wall clock, non-trivial memory. The hook's regex check on the happy path costs ~milliseconds.\n- **Trust boundary.** Both paths act only on URLs the agent already chose to send to its fetch tool. They do not introduce a new way for the agent to reach the internet — same URL surface, just a more capable backend.\n\n## Limitations\n\n- Cloudflare's hardest challenges (interactive Turnstile, etc.) may still defeat headless mode — flip `headless=False` in `cloak_fetch.py` if you need full CF coverage.\n- The hook reads `tool_response` as a string for regex matching. If a future Claude Code version changes the payload shape, the matcher's `jq` selector needs updating.\n- `additionalContext` size is bounded by Claude Code's hook output handling — very large pages are persisted to disk and only previewed inline (the persisted file path is shown so the agent can `Read` it).\n- The skill is **reactive**: it works only when the agent recognises the failure and recalls the skill. If the agent gives up after the first 403 without trying again, the skill doesn't help. The SKILL.md description is intentionally pushy to combat this — review and tweak if your agent under-triggers.\n\n## 💬 Community\n\n- **Discord:** https:\u002F\u002Fdiscord.gg\u002F79JF5Atuk\n- **WeChat:** scan the QR code below\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FAgents365-ai\u002Fimages_payment\u002Fmain\u002Fqrcode\u002Fagents365ai_wechat_1.png\" width=\"200\" alt=\"WeChat Community Group\">\n\u003C\u002Fp>\n\n## ❤️ Support\n\nIf cloakFetch saves you from one more \"HTTP 403 Forbidden\", consider supporting the author:\n\n\u003Ctable>\n  \u003Ctr>\n    \u003Ctd align=\"center\">\n      \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FAgents365-ai\u002Fimages_payment\u002Fmain\u002Fqrcode\u002Fwechat-pay.png\" width=\"180\" alt=\"WeChat Pay\">\n      \u003Cbr>\n      \u003Cb>WeChat Pay\u003C\u002Fb>\n    \u003C\u002Ftd>\n    \u003Ctd align=\"center\">\n      \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FAgents365-ai\u002Fimages_payment\u002Fmain\u002Fqrcode\u002Falipay.png\" width=\"180\" alt=\"Alipay\">\n      \u003Cbr>\n      \u003Cb>Alipay\u003C\u002Fb>\n    \u003C\u002Ftd>\n    \u003Ctd align=\"center\">\n      \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FAgents365-ai\u002Fimages_payment\u002Fmain\u002Fqrcode\u002Fbuymeacoffee.png\" width=\"180\" alt=\"Buy Me a Coffee\">\n      \u003Cbr>\n      \u003Cb>Buy Me a Coffee\u003C\u002Fb>\n    \u003C\u002Ftd>\n    \u003Ctd align=\"center\">\n      \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FAgents365-ai\u002Fimages_payment\u002Fmain\u002Fawarding\u002Faward.gif\" width=\"180\" alt=\"Give a Reward\">\n      \u003Cbr>\n      \u003Cb>Give a Reward\u003C\u002Fb>\n    \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n## 👤 Author\n\n**Agents365-ai**\n\n- GitHub: https:\u002F\u002Fgithub.com\u002FAgents365-ai\n- Bilibili: https:\u002F\u002Fspace.bilibili.com\u002F441831884\n\n## 📄 License\n\n[MIT](LICENSE)\n","cloakFetch 是一个用于处理被 Cloudflare 保护的 URL 的工具，能够自动绕过 Cloudflare 的反爬虫机制并返回干净的 Markdown 内容。该项目使用 Python 编写，通过与 CloakBrowser 和 trafilatura 结合，当 WebFetch 请求遇到 403 错误时，可以透明地重试并通过无头浏览器获取内容。其核心功能包括自动化的 Claude Code PostToolUse 钩子和手动调用的 SKILL.md 技能两种激活路径，前者无需用户干预即可完成请求重试，后者则需要代理在检测到特定错误模式后主动触发。此项目适用于需要从受 Cloudflare 保护的网站抓取数据的应用场景，如新闻聚合、学术研究资料收集等。","2026-06-11 04:02:33","CREATED_QUERY"]