[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-73228":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":23,"hasPages":23,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":43,"readmeContent":44,"aiSummary":45,"trendingCount":16,"starSnapshotCount":16,"syncStatus":46,"lastSyncTime":47,"discoverSource":48},73228,"semantic-router","vllm-project\u002Fsemantic-router","vllm-project","System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge","https:\u002F\u002Fvllm-semantic-router.com",null,"Go",4309,704,58,96,0,27,62,159,81,30.54,"Apache License 2.0",false,"main",[26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,5,42],"ai-gateway","bert-classification","fine-tuning","golang","huggingface-candle","huggingface-transformers","kubernetes","llm","llmrouter","mcp","mixture-of-models","openclaw","pii-detection","prompt-engineering","prompt-guard","rust","vllm","2026-06-12 02:03:10","\u003Cdiv align=\"center\">\n\n\u003Cimg src=\"website\u002Fstatic\u002Fimg\u002Fartworks\u002Fvllm-sr-logo.dark.png\" alt=\"vLLM Semantic Router\" width=\"50%\"\u002F>\n\n\u003Cp>\u003Cstrong>System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge\u003C\u002Fstrong>\u003C\u002Fp>\n\n\u003Cp>\n  \u003Ca href=\"https:\u002F\u002Fvllm-semantic-router.com\">Documentation\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fplay.vllm-semantic-router.com\">Playground\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fvllm-semantic-router.com\u002Fblog\u002F\">Blog\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fvllm-semantic-router.com\u002Fpublications\u002F\">Publications\u003C\u002Fa> |\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002FLLM-Semantic-Router\">Hugging Face\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003C\u002Fdiv>\n\n---\n\n## About\n\nIn the LLM era, the number of models is exploding. Different models vary across capability, scale, cost, and privacy boundaries. Choosing and connecting the right models to build semantic AI infrastructure is a system problem.\n\n**vLLM Semantic Router** is a **signal-driven** intelligent router for that problem. It helps teams build model systems that are more **efficient**, **safer**, and more **adaptive** across cloud, data center, and edge environments.\n\n![system](website\u002Fstatic\u002Fimg\u002Fsystem.png)\n\nIt delivers three core values:\n\n- **Token economics**: reduce wasted tokens, increase effective output, and maximize the value of every token.\n- **LLM safety**: detect jailbreaks, sensitive leakage, and hallucinations so agents remain controllable, trustworthy, and auditable.\n- **Fullmesh intelligence**: build personal AI at the edge and intelligent MaaS in the cloud by coordinating local, private, and frontier models across cost, privacy, and capability boundaries.\n\n## Getting Started\n\n### Install\n\n```bash\ncurl -fsSL https:\u002F\u002Fvllm-semantic-router.com\u002Finstall.sh | bash\n```\n\nFor platform notes, detailed setup options, and troubleshooting, see the **[Installation Guide](https:\u002F\u002Fvllm-semantic-router.com\u002Fdocs\u002Finstallation\u002F)**.\n\n> [!IMPORTANT]\n> Online [playground](https:\u002F\u002Fplay.vllm-semantic-router.com) default credentials:\n>\n> \u003C!-- markdownlint-disable MD004 MD032 -->\n> + username: `love@vllm-sr.ai`\n> + password: `vllm-sr`\n> \u003C!-- markdownlint-enable MD004 MD032 -->\n\n## Latest News\n\n- [2026\u002F03\u002F24] Vision Paper Released: [The Workload-Router-Pool Architecture for LLM Inference Optimization](https:\u002F\u002Fvllm-semantic-router.com\u002Fvision-paper)\n- [2026\u002F03\u002F10] v0.2 Released: [vLLM Semantic Router v0.2 Athena Release](https:\u002F\u002Fvllm.ai\u002Fblog\u002Fv0.2-vllm-sr-athena-release)\n- [2026\u002F02\u002F27] White Paper Released: [Signal Driven Decision Routing for Mixture-of-Modality Models](https:\u002F\u002Fvllm-semantic-router.com\u002Fwhite-paper\u002F)\n- [2026\u002F01\u002F05] Iris v0.1 Released: [vLLM Semantic Router v0.1 Iris: The First Major Release](https:\u002F\u002Fblog.vllm.ai\u002F2026\u002F01\u002F05\u002Fvllm-sr-iris.html)\n- [2025\u002F12\u002F16] Collaboration: [AMD × vLLM Semantic Router: Building the System Intelligence Together](https:\u002F\u002Fblog.vllm.ai\u002F2025\u002F12\u002F16\u002Fvllm-sr-amd.html)\n- [2025\u002F11\u002F19] New Blog: [Signal-Decision Driven Architecture: Reshaping Semantic Routing at Scale](https:\u002F\u002Fblog.vllm.ai\u002F2025\u002F11\u002F19\u002Fsignal-decision.html)\n- [2025\u002F11\u002F03] Paper Published: [Category-Aware Semantic Caching for Heterogeneous LLM Workloads](https:\u002F\u002Farxiv.org\u002Fabs\u002F2510.26835)\n- [2025\u002F10\u002F12] Paper Accepted: [When to Reason: Semantic Router for vLLM](https:\u002F\u002Farxiv.org\u002Fabs\u002F2510.08731)\n\n\u003Cdetails>\n\u003Csummary>Earlier announcements\u003C\u002Fsummary>\n\n- [2025\u002F12\u002F15] New Blog: [Token-Level Truth: Real-Time Hallucination Detection for Production LLMs](https:\u002F\u002Fblog.vllm.ai\u002F2025\u002F12\u002F14\u002Fhalugate.html)\n- [2025\u002F10\u002F27] New Blog: [Scaling Semantic Routing with Extensible LoRA](https:\u002F\u002Fblog.vllm.ai\u002F2025\u002F10\u002F27\u002Fsemantic-router-modular.html)\n- [2025\u002F10\u002F08] Collaboration: vLLM Semantic Router with [vLLM Production Stack](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fproduction-stack) Team.\n- [2025\u002F09\u002F01] Released the project: [vLLM Semantic Router: Next Phase in LLM inference](https:\u002F\u002Fblog.vllm.ai\u002F2025\u002F09\u002F11\u002Fsemantic-router.html).\n\n\u003C\u002Fdetails>\n\nMore announcements are available on the **[Blog](https:\u002F\u002Fvllm-semantic-router.com\u002Fblog\u002F)** and **[Publications](https:\u002F\u002Fvllm-semantic-router.com\u002Fpublications\u002F)** pages.\n\n## Community\n\nFor questions, feedback, or to contribute, please join the `#semantic-router` channel in vLLM Slack.\n\n### Community Meetings\n\nWe host bi-weekly community meetings to sync with contributors across different time zones:\n\n- **First Tuesday of the month**: 9:00-10:00 AM EST (accommodates US EST, EU, and Asia Pacific contributors)\n  - [Zoom Link](https:\u002F\u002Fus05web.zoom.us\u002Fj\u002F84122485631?pwd=BB88v03mMNLVHn60YzVk4PihuqBV9d.1)\n  - [Google Calendar Invite](https:\u002F\u002Fus05web.zoom.us\u002Fmeeting\u002FtZAsdeuspj4sGdVraOOR4UaXSstrH2jjPYFq\u002Fcalendar\u002Fgoogle\u002Fadd?meetingMasterEventId=4jjzUKSLSLiBHtIKZpGc3g)\n  - [ics file](https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F15wO8cg0ZjNxdr8OtGiZyAgkSS8_Wry0J\u002Fview?usp=sharing)\n- **Third Tuesday of the month**: 1:00-2:00 PM EST (accommodates US EST and California contributors)\n  - [Zoom Link](https:\u002F\u002Fus06web.zoom.us\u002Fj\u002F86871492845?pwd=LcTtXm9gtGu23JeWqXxbnLLCCvbumB.1)\n  - [Google Calendar Invite](https:\u002F\u002Fus05web.zoom.us\u002Fmeeting\u002FtZIlcOispzkiHtH2dlkWlLym68bEqvuf3MU5\u002Fcalendar\u002Fgoogle\u002Fadd?meetingMasterEventId=PqWz2vk7TOCszPXqconGAA)\n  - [ics file](https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F1T54mwYpXXoV9QfR76I56BFBPNbykSsTw\u002Fview?usp=sharing)\n- Meeting recordings: [YouTube](https:\u002F\u002Fwww.youtube.com\u002F@vLLMSemanticRouter\u002Fvideos)\n\n## Contributing\n\nIf you want to contribute, start with **[CONTRIBUTING.md](CONTRIBUTING.md)**.\n\nFor repository-native development workflow and validation commands, use **[AGENTS.md](AGENTS.md)** as the entrypoint and **[docs\u002Fagent\u002FREADME.md](docs\u002Fagent\u002FREADME.md)** as the canonical index.\n\n## Citation\n\nIf you find Semantic Router helpful in your research or projects, please consider citing it:\n\n```\n@misc{semanticrouter2025,\n  title={vLLM Semantic Router},\n  author={vLLM Semantic Router Team},\n  year={2025},\n  howpublished={\\url{https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fsemantic-router}},\n}\n```\n\n## Star History\n\n[![Star History Chart](https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=vllm-project\u002Fsemantic-router&type=Date)](https:\u002F\u002Fwww.star-history.com\u002F#vllm-project\u002Fsemantic-router&Date)\n\n## Sponsors\n\nWe are grateful to our sponsors who support us:\n\n---\n\n[**AMD**](https:\u002F\u002Fwww.amd.com) provides us with GPU resources and [ROCm™](https:\u002F\u002Fwww.amd.com\u002Fen\u002Fproducts\u002Fsoftware\u002Frocm.html) software for training and researching frontier router models, enhancing E2E testing, and building the online models playground.\n\n\u003Cdiv align=\"center\">\n\u003Ca href=\"https:\u002F\u002Fwww.amd.com\">\n  \u003Cimg src=\"website\u002Fstatic\u002Fimg\u002Famd-logo.svg\" alt=\"AMD\" width=\"40%\"\u002F>\n\u003C\u002Fa>\n\u003C\u002Fdiv>\n\n---\n","vLLM Semantic Router 是一个面向云、数据中心和边缘环境的系统级智能路由解决方案，用于混合模型。它基于信号驱动机制，帮助团队构建更高效、更安全且更具适应性的模型系统。其核心功能包括：通过减少无效令牌来优化成本（Token经济学）、增强大语言模型的安全性以防止越狱和敏感信息泄露，并通过协调不同成本、隐私及能力边界的本地、私有和前沿模型实现全网智能。适用于需要在多环境中部署复杂AI应用的企业或研究机构，特别是在对模型性能、安全性及资源利用效率有较高要求的场景下。",2,"2026-06-11 03:44:36","high_star"]