[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-902":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},902,"CrabTrap","brexhq\u002FCrabTrap","brexhq","An LLM-as-a-judge HTTP proxy to secure agents in production ","https:\u002F\u002Fwww.brex.com\u002Fcrabtrap",null,"Go",619,46,4,1,0,7,10,35,21,9.02,"MIT License",false,"main",true,[],"2026-06-12 02:00:20","# CrabTrap\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flogo.png\" alt=\"CrabTrap logo\" width=\"600\" \u002F>\n\u003C\u002Fp>\n\nAn HTTP\u002FHTTPS proxy that sits between AI agents and external APIs, evaluating every outbound request against security policies before it reaches the internet.\n\nIf you run AI agents that call external services — Slack, Gmail, GitHub, or anything else — CrabTrap gives you guardrails. It intercepts every outbound HTTP\u002FHTTPS request, checks it against deterministic rules and an LLM-based policy judge, and either forwards it or blocks it with a reason. Every request and decision is logged to PostgreSQL for a complete audit trail.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"docs\u002Fcrabtrap-flow.svg\" alt=\"CrabTrap request flow\" width=\"800\" \u002F>\n\u003C\u002Fp>\n\n## Quickstart\n\nCrabTrap runs as a Docker container alongside PostgreSQL. See [QUICKSTART.md](QUICKSTART.md) for the full walkthrough — the short version:\n\n```bash\ndocker compose up -d                                                    # start CrabTrap + Postgres\ndocker compose cp crabtrap:\u002Fapp\u002Fcerts\u002Fca.crt .\u002Fca.crt                   # copy the generated CA cert\n# create test-admin admin user and store their web_token in a variable\nadmin_token=$(docker compose exec -it crabtrap .\u002Fgateway create-admin-user test-admin \\\n    | tail -n1 | cut -d\" \" -f2)\ntoken=$(curl -X POST http:\u002F\u002Flocalhost:8081\u002Fadmin\u002Fusers \\\n    -H \"Content-Type: application\u002Fjson\" \\\n    -H \"Authorization: Bearer ${admin_token}\" \\\n    -d '{\"id\": \"alice@example.com\", \"is_admin\": false}' \\\n    | jq -r '.channels[] | select(.channel_type == \"gateway_auth\") | .gateway_auth_token')\n# test with\ncurl -x http:\u002F\u002F${token}:@localhost:8080 \\\n    --cacert ca.crt https:\u002F\u002Fhttpbin.org\u002Fget\n```\n\nThe proxy listens on `localhost:8080`, the admin UI is at `localhost:8081` and you can login to it with the $admin_token.\n\n\n## How It Works\n\n1. **Agent connects** — set `HTTP_PROXY` and `HTTPS_PROXY` to point at CrabTrap\n2. **TLS termination** — CrabTrap generates a per-host certificate from a custom CA and decrypts the request\n3. **Static rules** — the request is matched against URL pattern rules (prefix, exact, or glob). If a rule matches, the decision is immediate — no LLM call. Deny rules always take priority over allow.\n4. **LLM judge** — if no static rule matches, the request is evaluated by an LLM against the agent's natural-language security policy. Allowed requests are forwarded; denied requests get a 403 with the reason.\n5. **Audit logged** — every request, decision, and response is recorded in PostgreSQL\n\n## Features\n\n### Security\n\n- **HTTPS interception** — transparent MITM proxy with custom TLS server certificate generation\n- **SSRF protection** — blocks requests to private networks (RFC 1918, loopback, link-local, Carrier-Grade NAT, IPv6 ULA\u002FNAT64\u002F6to4) with DNS-rebinding prevention\n- **Prompt injection defense** — request payloads are JSON-encoded and policy content is JSON-escaped before being sent to the LLM judge\n- **Per-IP rate limiting** — token bucket rate limiter (default 50 req\u002Fs, burst 100)\n\n### Policy Evaluation\n\n- **Two-tier evaluation** — deterministic static rules are checked first; the LLM judge is only invoked if no rule matches\n- **Static rules** — prefix, exact, and glob URL pattern matching with optional HTTP method filters\n- **Per-agent LLM policies** — natural-language security policies evaluated via LLM\n- **Circuit breaker** — trips after 5 consecutive LLM failures, reopens after 10s cooldown\n- **Configurable fallback** — deny (default) or passthrough when the LLM judge is unavailable\n\n### Operations\n\n- **Policy builder** — an agentic loop that analyzes observed traffic and drafts security policies automatically\n- **Eval system** — replay historical audit log entries against a policy to measure accuracy\n- **Web UI** — audit trail viewer, policy editor, eval results, and agent management\n\n## What CrabTrap Does NOT Do\n\n- **Not a WAF or inbound firewall** — CrabTrap is a forward proxy (outbound-only) for agent-originated traffic. It does not inspect inbound requests to your services.\n- **Does not redact sensitive data** — the proxy sees all request content in cleartext, including headers like Authorization and Cookie. This is by design; the trust boundary is the proxy itself.\n- **Does not provide human-in-the-loop approval** — there is no approval queue, no Slack prompts, and no escalation path. Decisions are made automatically by static rules and the LLM judge.\n- **Does not filter API responses** — only outbound requests are evaluated. Responses from upstream APIs are streamed back to the agent unexamined.\n- **Does not inspect WebSocket frames** — only the WebSocket upgrade request is evaluated. Once upgraded, frames pass through uninspected.\n\n## Configuration\n\n| Section | Key Settings |\n|---|---|\n| `proxy` | Port (default 8080), timeouts, rate limits, SSRF CIDR allowlist |\n| `tls` | CA cert\u002Fkey paths, certificate cache size (default 10,000) |\n| `approval` | Mode: `llm` or `passthrough`, timeout (default 30s) |\n| `llm_judge` | Provider, model IDs, fallback mode (`deny`\u002F`passthrough`), circuit breaker |\n| `database` | PostgreSQL connection URL (supports `${DATABASE_URL}` expansion) |\n| `audit` | Output destination: `stderr` (default), `stdout`, or a file path |\n| `log_level` | `debug`, `info` (default), `warn`, `error` |\n\nSee [`config\u002Fgateway.yaml.example`](config\u002Fgateway.yaml.example) for the full reference with inline comments.\n\n## Project Structure\n\n```\ncrabtrap\u002F\n├── cmd\u002Fgateway\u002F          # Entry point, admin API wiring, web UI serving\n├── internal\u002F\n│   ├── proxy\u002F            # MITM proxy, TLS cert generation, SSRF protection, rate limiting\n│   ├── approval\u002F         # Static rules engine + approval orchestration\n│   ├── judge\u002F            # LLM judge prompt construction + response parsing\n│   ├── llm\u002F              # LLM adapters, circuit breaker, concurrency control\n│   ├── builder\u002F          # Policy agent (agentic loop with tools)\n│   ├── eval\u002F             # Eval system (replay audit entries against policies)\n│   ├── admin\u002F            # Admin API routes, auth, user\u002Faudit stores\n│   ├── llmpolicy\u002F        # Policy storage and versioning\n│   ├── audit\u002F            # Structured JSON logging + event dispatch\n│   ├── config\u002F           # YAML config loading, validation, defaults\n│   ├── db\u002F               # PostgreSQL connection pool + migrations\n│   └── notifications\u002F    # SSE channel + event dispatcher\n├── pkg\u002Ftypes\u002F            # Shared types (StaticRule, LLMPolicy, AuditEntry, etc.)\n├── web\u002Fsrc\u002F              # React + TypeScript admin UI (Vite)\n├── config\u002F               # YAML configuration files\n├── certs\u002F                # Generated TLS certificates (not committed)\n└── scripts\u002F              # Certificate generation, database migrations\n```\n\n## Observability\n\nCrabTrap ships an optional OpenTelemetry metric surface, exposed in Prometheus scrape format on a dedicated HTTP listener. Disabled by default.\n\nEnable it in `config\u002Fgateway.yaml`:\n\n```yaml\nobservability:\n  metrics:\n    enabled: true                # default: false\n    listen: \"127.0.0.1:9090\"     # bind address (default loopback)\n```\n\nThe metrics listener is separate from the admin and proxy ports. It serves only `\u002Fmetrics` and requires no auth — Prometheus scrapers do not need admin credentials. Network exposure is controlled by the `listen` bind address: the default `127.0.0.1:9090` keeps the surface on loopback so no operator action is required to keep it private. To scrape from another host, change `listen` to a private interface address (e.g. `10.0.1.42:9090`) and reach it from inside the trust boundary.\n\nMinimal Prometheus scrape config (loopback bind, scraper on the same host):\n\n```yaml\nscrape_configs:\n  - job_name: crabtrap\n    static_configs:\n      - targets: ['127.0.0.1:9090']\n    metrics_path: \u002Fmetrics\n    scrape_interval: 15s\n```\n\nThe current metric surface includes counters for rate-limit hits and approval decisions, a gauge per LLM provider for circuit-breaker state, histograms for judge and approval latency, and a `crabtrap_build_info` gauge that lets operators correlate metric anomalies with deployments. See [docs\u002Fobservability.md](docs\u002Fobservability.md) for the full metric catalog, alert suggestions, and cardinality notes.\n\n## Development\n\n```bash\nmake test          # lint (go vet + staticcheck) then tests with -race\nmake fmt           # format Go code\nmake lint          # go vet + staticcheck\nmake build         # production binary with embedded web UI\nmake build-web     # rebuild web UI only\n```\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for the full development workflow, PR guidelines, and coding conventions.\n\n## Releases\n\nReleases are automated with [GoReleaser](https:\u002F\u002Fgoreleaser.com\u002F) via GitHub Actions. Tag a commit on `main` and push:\n\n```bash\ngit tag v1.2.3\ngit push origin v1.2.3\n```\n\nThis builds cross-platform binaries (linux\u002Fdarwin, amd64\u002Farm64), creates a GitHub Release with a changelog, and pushes multi-arch Docker images to `quay.io\u002Fbrexhq\u002Fcrabtrap`.\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md#releasing-with-goreleaser) for release notes and commit message conventions.\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE).\n\n## Contributing\n\nWe welcome contributions! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on getting started, running tests, and submitting pull requests.\n\n## More\n\n- Troubleshooting: [TROUBLESHOOTING.md](TROUBLESHOOTING.md)\n- Architecture: [DESIGN.md](DESIGN.md)\n- Issues: https:\u002F\u002Fgithub.com\u002Fbrexhq\u002FCrabTrap\u002Fissues\n","CrabTrap 是一个用于保护生产环境中AI代理的HTTP\u002FHTTPS代理，它在AI代理和外部API之间拦截所有出站请求，并根据安全策略进行评估。项目核心功能包括基于静态规则和LLM（大语言模型）的安全策略判断，能够有效防止SSRF攻击、提示注入等安全威胁，并对每个请求及其决策结果记录到PostgreSQL数据库中，提供完整的审计追踪。适用于需要对外部服务调用进行严格控制与监控的AI应用环境，如集成Slack、Gmail或GitHub等服务的企业级AI解决方案。",2,"2026-06-11 02:40:06","CREATED_QUERY"]