[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-96018":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":9,"totalLinesOfCode":9,"stars":12,"forks":13,"watchers":14,"openIssues":14,"contributorsCount":9,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":14,"stars30d":14,"stars90d":14,"forks30d":14,"starsTrendScore":14,"compositeScore":15,"rankGlobal":9,"rankLanguage":9,"license":9,"archived":16,"fork":16,"defaultBranch":17,"hasWiki":16,"hasPages":16,"topics":9,"createdAt":9,"pushedAt":9,"updatedAt":18,"readmeContent":19,"aiSummary":20,"trendingCount":14,"starSnapshotCount":14,"syncStatus":21,"lastSyncTime":9,"discoverSource":22},96018,"agentic-api","vllm-project\u002Fagentic-api","vllm-project","Stateful API logic for agentic applications using vLLM",null,"https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fagentic-api","Rust",198,56,0,42.27,false,"main","2026-09-20 04:01:32","\u003Cdiv align=\"center\">\n\n\u003Cpicture>\n  \u003Csource media=\"(prefers-color-scheme: dark)\" srcset=\"assets\u002FWhite-Main-Logo.svg\">\n  \u003Csource media=\"(prefers-color-scheme: light)\" srcset=\"assets\u002FBlack-Main-Logo.svg\">\n  \u003Cimg alt=\"Agentic API\" src=\"assets\u002FBlack-Main-Logo.svg\" width=\"600\">\n\u003C\u002Fpicture>\n\n**The stateful, agentic API layer for [vLLM](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm), written in Rust 🦀**\n\n*Run OpenAI-grade agentic workloads (Responses API, server-side tools, Codex) on your own GPUs.*\n\n[![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-Apache%202.0-blue.svg)](LICENSE)\n[![Rust](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FRust-1.85%2B-orange.svg?logo=rust)](Cargo.toml)\n[![CI](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fagentic-api\u002Factions\u002Fworkflows\u002Frust.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fagentic-api\u002Factions\u002Fworkflows\u002Frust.yml)\n[![pre-commit](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fagentic-api\u002Factions\u002Fworkflows\u002Fpre-commit.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fagentic-api\u002Factions\u002Fworkflows\u002Fpre-commit.yml)\n\n\u003C\u002Fdiv>\n\n______________________________________________________________________\n\n## 🧠 Overview\n\nvLLM gives you state-of-the-art inference throughput. But real agentic applications need more than raw tokens: they need **conversation state, tool-call loops, and multi-turn orchestration**. Today, all of that complexity lives in your client code.\n\n**Agentic API moves it server-side.** It is a Rust-native gateway that sits in front of vLLM and owns the stateful agentic APIs, starting with an OpenAI-compatible [Responses API](https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fapi-reference\u002Fresponses). vLLM is one supported backend, not part of the Agentic API product name. Your application makes *one API call* and the server handles the rest: state hydration, tool execution, streaming, and continuation.\n\n```mermaid\nflowchart LR\n    C([\"🧑‍💻 Client\u003Cbr\u002F>Codex · SDKs · curl\"]) -->|\"📮 \u003Ccode>POST \u002Fv1\u002Fresponses\u003C\u002Fcode>\u003Cbr\u002F>🌐 HTTP  📡 SSE  🔌 WebSocket\"| A\n    subgraph A [\"⚡ Agentic API (Rust 🦀)\"]\n        direction TB\n        S[\"🔄 State hydration\u003Cbr\u002F>\u003Ccode>previous_response_id\u003C\u002Fcode>\"]\n        T[\"🛠️ Server-side tools\u003Cbr\u002F>web search · functions\"]\n        P[\"💾 Persistence\u003Cbr\u002F>SQLite response store\"]\n    end\n    A -->|\"🚀 \u003Ccode>POST \u002Fv1\u002Fresponses\u003C\u002Fcode>\u003Cbr\u002F>⚙️ stateless  🤝 OpenAI-compatible\"| V([\"🚀 vLLM core\u003Cbr\u002F>inference engine\"])\n\n    classDef client fill:#FFE8B3,stroke:#F59E0B,stroke-width:2px,color:#7C2D12\n    classDef inner fill:#E0E7FF,stroke:#6366F1,stroke-width:2px,color:#312E81\n    classDef engine fill:#DCFCE7,stroke:#22C55E,stroke-width:2px,color:#14532D\n\n    class C client\n    class S,T,P inner\n    class V engine\n    style A fill:#F5F3FF,stroke:#8B5CF6,stroke-width:2px,color:#5B21B6\n    linkStyle 0 stroke:#F59E0B,stroke-width:2px\n    linkStyle 1 stroke:#22C55E,stroke-width:2px\n```\n\n> [!TIP]\n> Point [OpenAI Codex](https:\u002F\u002Fgithub.com\u002Fopenai\u002Fcodex) at Agentic API and drive it entirely with open models served by vLLM. No OpenAI account required.\n\n## ✨ Key Features\n\n- 🔄 **Stateful conversations**: the server manages history via `previous_response_id`. No client-side message tracking, no replaying full transcripts.\n- 🛠️ **Server-side tool execution**: an explicit tool-ownership model (gateway \u002F client \u002F provider) decides exactly what runs where. Web search ships today via [You.com](https:\u002F\u002Fyou.com), and the model executes multi-step tool chains automatically.\n- 📡 **Every transport**: non-streaming HTTP, server-sent events for token streaming, and full **WebSocket** support for interactive clients.\n- 🧰 **Codex-ready**: accepts Codex-shaped Responses traffic out of the box, preserving the tool declarations and response item shapes Codex depends on.\n- 🏃 **Background execution**: fire-and-forget requests that keep processing server-side.\n- ✅ **Compatibility tested**: validated against the [Open Responses](https:\u002F\u002Fwww.openresponses.org\u002F) compatibility suite, with replay-cassette tests for real OpenAI and vLLM traffic.\n\n## 🧭 API Surface\n\n| Endpoint | Description | Status |\n| --- | --- | --- |\n| `POST \u002Fv1\u002Fresponses` | OpenAI-compatible Responses API with state, tools, and streaming | ✅ |\n| `GET \u002Fv1\u002Fresponses` | WebSocket transport for the Responses API | ✅ |\n| `POST \u002Fv1\u002Fconversations` | Conversation management | ✅ |\n| `GET \u002Fv1\u002Fmodels` | Model listing proxied from vLLM | ✅ |\n| `GET \u002Fhealth` · `GET \u002Fready` | Liveness and readiness probes | ✅ |\n| Messages API | Anthropic-style stateful messages on shared primitives | 🚧 Planned |\n| Interactions API | Higher-level agentic workflow surface | ⏳ Planned |\n\n## 🚀 Quickstart\n\n### Agentic API CLI\n\nBuild the user-facing CLI and gateway binaries together:\n\n```bash\ncargo build -p agentic-server --bins\n```\n\nLaunch Codex or Claude Code with an isolated Agentic API configuration:\n\n```bash\n.\u002Ftarget\u002Fdebug\u002Fagentic run codex --model Qwen\u002FQwen3-30B-A3B-FP8\n.\u002Ftarget\u002Fdebug\u002Fagentic run claude --model Qwen\u002FQwen3-30B-A3B-FP8\n```\n\nTo use an existing upstream, provide its `http:\u002F\u002F` or `https:\u002F\u002F` base URL. The harness model defaults to the first\nmodel the upstream lists at `\u002Fv1\u002Fmodels`; pass `--model` to choose a different one:\n\n```bash\n.\u002Ftarget\u002Fdebug\u002Fagentic run codex --upstream http:\u002F\u002F127.0.0.1:5050\n.\u002Ftarget\u002Fdebug\u002Fagentic run claude \\\n  --upstream http:\u002F\u002F127.0.0.1:5050 \\\n  --model Qwen\u002FQwen3-30B-A3B-FP8\n```\n\nSQLite is the default storage backend. Use PostgreSQL explicitly when the session is shared:\n\n```bash\n.\u002Ftarget\u002Fdebug\u002Fagentic run codex \\\n  --model Qwen\u002FQwen3-30B-A3B-FP8 \\\n  --database-url postgresql:\u002F\u002Fuser:password@localhost\u002Fagentic_api\n```\n\nRun preflight checks without launching a harness:\n\n```bash\n.\u002Ftarget\u002Fdebug\u002Fagentic validate \\\n  --upstream http:\u002F\u002F127.0.0.1:5050 \\\n  --model Qwen\u002FQwen3-30B-A3B-FP8 \\\n  --harness codex\n```\n\nUse `AGENTIC_CODEX_BIN` or `AGENTIC_CLAUDE_BIN` to override harness binary discovery. Add `--no-color` for scripts or\n`--quiet` for minimal lifecycle output. Use `--yolo` only in an externally isolated environment; it skips Claude\npermission checks and disables Codex approvals and sandboxing.\n\n### Python distribution\n\nThe `agentic-api` wheel packages the Rust gateway and a small Python launcher. This release produces wheel artifacts\nfor 0.5.0 as a build-only release: download the wheel for your platform from the release workflow, then install that local file. It is\nnot published on PyPI yet.\n\n```bash\nWHEEL_PATH=\u002Fabsolute\u002Fpath\u002Fto\u002Fagentic_api-PLATFORM.whl\nuv pip install \"$WHEEL_PATH\"\nagentic-api serve --vllm-base-url http:\u002F\u002Fexisting-vllm:8000\n\nuv pip install \"agentic-api[local] @ file:\u002F\u002F$WHEEL_PATH\"\nagentic-api serve --model MODEL_ID\n```\n\nThe base install is for remote mode and does not install vLLM. The `[local]` extra installs the pinned vLLM runtime so\nthe launcher can manage a local vLLM process on supported Linux hosts.\n\nUse `agentic-api --version` for a quick install check and `agentic-api doctor --mode remote --json` when an agent or\nscript needs machine-readable diagnostics.\n\n#### After PyPI publication\n\nThese public-index and `uvx` examples apply only after the PyPI publication gate for a future release:\n\n```bash\nuv pip install agentic-api\nuv pip install \"agentic-api[local]\"\nuvx --from agentic-api agentic-api doctor\nuvx --from agentic-api agentic-api serve --vllm-base-url http:\u002F\u002Fexisting-vllm:8000\n```\n\nThe Rust-native `agentic` CLI remains supported for `run codex`, `run claude`, `serve`, and `validate`. For the full\ninstallation walkthrough, managed-vLLM passthrough examples, `doctor` output, and known-good model profiles, see\n[Python installation and workflows](docs\u002Fguides\u002Fpython-installation.md).\n\nFor Claude sessions, Agentic API always sets both `--effort medium` and `CLAUDE_CODE_EFFORT_LEVEL=medium`; the\nenvironment variable is intentional because Claude Code gives it precedence over the command-line effort flag.\nQwen3.8-27B's vLLM chat template accepts `low`, `medium`, and `xhigh` reasoning effort values but not Claude Code's\ndefault `high`. Override the pinned value with `AGENTIC_CLAUDE_EFFORT`. See the [Claude Code effort\nconfiguration](https:\u002F\u002Fcode.claude.com\u002Fdocs\u002Fen\u002Fmodel-config) and [vLLM reasoning output\ndocumentation](https:\u002F\u002Fdocs.vllm.ai\u002Fen\u002Flatest\u002Ffeatures\u002Freasoning_outputs\u002F) for the underlying behavior, and\n[Harness CLI Testing](docs\u002Fguides\u002Fharness-cli-testing.md) for an end-to-end verification checklist.\n\n**1. Serve a model with vLLM.** Any recipe from [recipes.vllm.ai](https:\u002F\u002Frecipes.vllm.ai) works:\n\n```bash\nvllm serve Qwen\u002FQwen3-30B-A3B-FP8 \\\n  --tool-call-parser qwen3_coder --enable-auto-tool-choice \\\n  --reasoning-parser qwen3 --port 5050\n```\n\nServing through [NVIDIA Dynamo](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fdynamo) instead of a standalone `vllm serve`? Point the\ngateway at the Dynamo frontend the same way; see\n[Running Agentic API in front of NVIDIA Dynamo](docs\u002Fguides\u002Fdynamo-upstream.md).\n\n**2. Start Agentic API**, pointing it at the vLLM server (set the `YOU_*` variables to enable built-in web search):\n\n```bash\nYOU_API_KEY=\u003Cyour-you.com-api-key> YOU_API_BASE_URL=\u003Cyou.com-api-base-url> \\\n  cargo run -p agentic-server -- --llm-api-base http:\u002F\u002F0.0.0.0:5050\n```\n\nThe default database is `~\u002F.agentic-api\u002Fagentic_api.db`, so running an installed binary does not create state in the\ncurrent directory. Set `AGENTIC_API_HOME` to an absolute directory to move both the default database and user\nconfiguration, or set `DATABASE_URL`\u002F`--db-url` to select a different database.\n\n**3. Make a stateful call:**\n\n```bash\ncurl http:\u002F\u002Flocalhost:9000\u002Fv1\u002Fresponses \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"Qwen\u002FQwen3-30B-A3B-FP8\",\n    \"input\": \"What is new in vLLM this month?\",\n    \"tools\": [{\"type\": \"web_search\"}]\n  }'\n```\n\nContinue the conversation by passing the returned `id` as `previous_response_id`, and the server rehydrates everything for you.\n\n## ⚙️ Agentic API home and configuration\n\nOn startup, Agentic API creates `~\u002F.agentic-api` and loads `~\u002F.agentic-api\u002Fconfig.toml` when that file exists. On the\nfirst invocation with a resolved LLM base URL, a missing config file is generated from the effective\n`--llm-api-base`\u002F`LLM_API_BASE` and non-secret tool settings. It records the name of the web-search API-key environment\nvariable, never its value. The generated file is group-readable but not group-writable on Unix (so a container restart\nunder a different arbitrary UID sharing the same group can still read it) and is never overwritten on later\nruns. CLI arguments and process environment variables take precedence over file settings. A standalone server can\ntherefore be started with just `agentic-server` after creating a config like this:\n\n```toml\nllm_api_base = \"http:\u002F\u002F127.0.0.1:5050\"\n# database_url = \"postgresql:\u002F\u002Fagentic-api@localhost\u002Fagentic_api\"\n\n[web_search]\nbase_url = \"https:\u002F\u002Fapi.ydc-index.io\"\napi_key_env = \"YOU_API_KEY\"\n\n[mcp]\nallowed_hosts = [\"mcp.example.com\"]\n\n[tools]\n# Upper bound for gateway-owned calls within one Responses round and for\n# provider requests inside a batched web-search call.\n# Must be greater than zero.\nmax_concurrent_gateway_calls = 5\n\n[mcp_servers.counter]\nurl = \"https:\u002F\u002Fmcp.example.com\u002Fmcp\"\nallowed_tools = [\"tool_1_name\", \"tool_2_name\"]\nrequire_approval = \"never\"\n```\n\n`api_key_env` names the process environment variable containing the web-search credential; it does not contain the\ncredential itself. `YOU_API_BASE_URL`, `AGENTIC_MCP_ALLOWED_HOSTS`, and\n`AGENTIC_MAX_CONCURRENT_GATEWAY_CALLS` can override their typed file settings. The concurrency value is a sliding-window\nupper bound; handlers may further serialize calls to the same tool name. The MCP allowlist is used only for\nrequest-declared remote MCP URLs; configured `[mcp_servers]` entries are trusted operator configuration.\n\nWith that file in place, inject only the secret when starting the server:\n\n```bash\nYOU_API_KEY=\"\u003Cyour-you.com-api-key>\" agentic-server\n```\n\nRestrict the file to the service account (for example, `chmod 600 ~\u002F.agentic-api\u002Fconfig.toml`), especially if you add\ncredentialed `database_url`, MCP headers, or stdio MCP environment values. Prefer `DATABASE_URL`, referenced API-key\nenvironment variables, and a deployment secret manager for secrets.\n\nTo use an operator-configured MCP server, declare its label without sending its connection details or secrets:\n\n```json\n{\n  \"type\": \"mcp\",\n  \"server_label\": \"server_label\"\n}\n```\n\nConfigured `allowed_tools` form the maximum tool set; request-provided `allowed_tools` may narrow it. A configured\n`require_approval = \"never\"` lets requests omit that field. If a label exists in `config.toml`, a request cannot\noverride it with `server_url`; otherwise the existing request-declared HTTP MCP flow remains available.\n\n## 🤖 Codex on your own GPUs\n\nAgentic API speaks the Responses wire protocol Codex expects, including WebSockets, so you can run the full Codex experience against open models.\n\nAdd a provider to `~\u002F.codex\u002Fconfig.toml`:\n\n```toml\n[model_providers.agentic-api]\nname = \"OpenAI\"\nbase_url = \"http:\u002F\u002Flocalhost:9000\u002Fv1\"\nwire_api = \"responses\"\nrequires_openai_auth = false\nsupports_websockets = true\n```\n\nThen launch Codex:\n\n```bash\ncodex --disable image_generation -c model_provider=agentic-api -m Qwen\u002FQwen3-30B-A3B-FP8\n```\n\nIf the gateway enables OIDC, configure Codex's supported command-backed bearer authentication instead of\n`requires_openai_auth = false`:\n\n```toml\n[model_providers.agentic-api]\nname = \"OpenAI\"\nbase_url = \"http:\u002F\u002Flocalhost:9000\u002Fv1\"\nwire_api = \"responses\"\nsupports_websockets = true\n\n[model_providers.agentic-api.auth]\ncommand = \"\u002Fabsolute\u002Fpath\u002Fto\u002Fprint-oidc-token\"\nargs = [\"--audience\", \"agentic-api\"]\nrefresh_interval_ms = 300000\n```\n\nThe command must print only a current OIDC token to stdout. Codex refreshes it before expiry and sends it as the\nprovider bearer token. See the\n[Codex custom-provider authentication reference](https:\u002F\u002Fdevelopers.openai.com\u002Fcodex\u002Fconfig-advanced#custom-model-providers).\nKeep the inference credential in the gateway's `OPENAI_API_KEY`; do not print that service credential from the token\ncommand.\nSee [GitHub authentication with Dex](docs\u002Fdeploying\u002Fgithub-oidc.md) for a complete GitHub login, token-helper, and\ngateway setup.\n\n## 🧑‍💻 Claude Code on your own GPUs\n\nAgentic API serves the Anthropic Messages protocol at `\u002Fv1\u002Fmessages`, so Claude Code (CLI or Agent SDK) runs against open models. Point it at the gateway:\n\n```bash\nexport ANTHROPIC_BASE_URL=\"http:\u002F\u002Flocalhost:9000\"\nexport ANTHROPIC_API_KEY=\"\u003Cyour-key>\"\nexport ANTHROPIC_MODEL=\"Qwen\u002FQwen3-30B-A3B-FP8\"   # match the served model\n\nclaude -p \"summarize the files in this directory\"\n```\n\nWith OIDC enabled, use Claude Code's bearer-token variable and leave its API-key variable unset so the identity token\nis not also sent as an upstream `x-api-key`:\n\n```bash\nexport ANTHROPIC_BASE_URL=\"http:\u002F\u002Flocalhost:9000\"\nexport ANTHROPIC_AUTH_TOKEN=\"$(\u002Fabsolute\u002Fpath\u002Fto\u002Fprint-oidc-token --audience agentic-api)\"\nunset ANTHROPIC_API_KEY\n\nclaude -p \"summarize the files in this directory\"\n```\n\nRefresh `ANTHROPIC_AUTH_TOKEN` before it expires. For supported dynamic credential helpers, see Anthropic's\n[LLM gateway authentication guide](https:\u002F\u002Fdocs.anthropic.com\u002Fen\u002Fdocs\u002Fclaude-code\u002Fllm-gateway).\nThe same [GitHub authentication with Dex](docs\u002Fdeploying\u002Fgithub-oidc.md) guide shows how to obtain the ID token\nwithout embedding a client secret in Claude Code.\n\nClaude Code's own tools (Bash, Edit, Read, …) stay **client-owned** — Claude Code runs them, as usual.\n\n### Running Claude Code's web search on the gateway\n\nCurrent Claude Code versions declare Anthropic's native `web_search_20250305` server tool. Agentic API translates that\ndeclaration for the upstream model and executes the resulting search server-side against the configured search backend;\nno MCP server or tool alias is required:\n\n```bash\nYOU_API_KEY=\u003Cyou.com-key> YOU_API_BASE_URL=\u003Cyou.com-base-url> \\\n  cargo run -p agentic-server -- --llm-api-base http:\u002F\u002F0.0.0.0:5050\n```\n\nThe gateway supports the basic `web_search_20250305` contract, including `max_uses`, `allowed_domains`,\n`blocked_domains`, and the country in `user_location`. Other versioned native web-search declarations are rejected rather\nthan forwarded in a shape the upstream cannot execute.\n\nOlder clients that declare a function tool named `WebSearch` can still opt in with\n`MESSAGES_GATEWAY_TOOL_ALIASES=\"WebSearch=web_search\"`. This variable maps a client tool name to a gateway executor\n(`name=executor`, comma-separated) and remains empty by default. The gateway adapts the older `WebSearch` function's\n`allowed_domains`\u002F`blocked_domains` arguments to the executor's schema automatically.\n\n> Note: allow and block domain lists are mutually exclusive, matching Anthropic's native tool contract.\n\n## 🧩 Tool Ownership Model\n\nEvery tool call has exactly one execution path, so nothing runs by accident:\n\n| Ownership | Who executes it | Examples |\n| --- | --- | --- |\n| **Gateway-owned** | Agentic API executes it server-side and continues the loop | Web search, file search, MCP-backed tools |\n| **Client-owned** | Preserved and returned to the client | Codex shell \u002F editor tools, your functions |\n| **Provider-owned** | Passed through to vLLM or an upstream provider | Provider-native tools |\n\nUnknown or ambiguous tool shapes are **never executed by default**. They are preserved and returned.\n\n## 🏗️ Repository Layout\n\n```\ncrates\u002F\n├── agentic-server\u002F       # Axum binary, transport handlers (HTTP\u002FSSE\u002FWS), configuration\n├── agentic-server-core\u002F  # Protocol types, executor, tool framework, persistence\n└── agentic-praxis\u002F       # Praxis gateway integration\ndocs\u002F                     # MkDocs documentation, ADRs, and design notes\n```\n\n## 🛠️ Developing\n\n```bash\ncargo build                                  # build\ncargo test                                   # test\ncargo clippy --all-targets -- -D warnings    # lint\ncargo fmt -- --check                         # format check\n```\n\nDocs are built with MkDocs:\n\n```bash\nuv venv\nuv pip install -r docs\u002Frequirements.txt\nuv run mkdocs serve\n```\n\nDesign and migration decisions are tracked as ADRs in [docs\u002Fadr\u002F](docs\u002Fadr\u002F), with deeper design notes in [docs\u002Fdesign\u002F](docs\u002Fdesign\u002F). See the full [ROADMAP](ROADMAP.md) for where the project is heading, and [CONTRIBUTING](CONTRIBUTING.md) to get involved.\n\n## 🗺️ Roadmap at a Glance\n\n- [x] **Responses API hydration**: stateful continuation with `previous_response_id`\n- [x] **Codex support**: practical Codex sessions through the Responses API\n- [x] **Server-side tool execution**: explicit ownership, web search built in\n- [ ] **Messages API**: built on the same persistence and execution primitives\n- [ ] **Interactions API**: durable, higher-level agentic workflows\n- [ ] **Production hardening**: storage backends, observability, cached-prefix continuation\n\n______________________________________________________________________\n\n## 📄 License\n\nLicensed under the [Apache License 2.0](LICENSE).\n\n\u003Cdiv align=\"center\">\n\n**⭐ If Agentic API saves you from writing one more client-side tool loop, star the repo! ⭐**\n\n\u003C\u002Fdiv>\n","Agentic API 是一个基于 Rust 构建的有状态服务层，为 vLLM 推理引擎提供面向智能体（agentic）应用的后端支持。它封装了多轮对话状态管理、工具调用循环、响应持久化（SQLite）、流式传输（SSE\u002FWebSocket）及 OpenAI 兼容的 Responses API，使客户端可通过单次请求触发完整工具增强型推理流程。项目专为需长期上下文维护、自主工具执行与服务端编排能力的 AI 应用场景设计，如本地部署的 AI 助手、自动化工作流引擎和开源 Codex 替代方案。",2,"trending"]