[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92526":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":15,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":17,"rankGlobal":9,"rankLanguage":9,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":9,"pushedAt":9,"updatedAt":32,"readmeContent":33,"aiSummary":34,"trendingCount":15,"starSnapshotCount":15,"syncStatus":35,"lastSyncTime":36,"discoverSource":37},92526,"nexus-llm-router","Francis1998\u002Fnexus-llm-router","Francis1998","Intelligent multi-LLM router with task-aware routing strategies, cost optimization, and production safety controls — drop-in OpenAI-compatible API",null,"Python",69,8,4,1,0,17,44.56,"Other",false,"main",true,[23,24,25,26,27,28,29,30,31],"ai-agents","ai-infrastructure","cost-optimization","llm-gateway","llm-router","mlops","multi-llm","openai-compatible","routing","2026-07-22 04:02:06","# nexus-llm-router\n\n![Tests](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Ftests-85%20passing-brightgreen) ![Python](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpython-3.11%2B-blue) ![CI](https:\u002F\u002Fgithub.com\u002FFrancis1998\u002Fnexus-llm-router\u002Factions\u002Fworkflows\u002Fci.yml\u002Fbadge.svg)\n\n> Intelligent multi-LLM routing middleware with task-aware model selection, cost optimization, fallback safety, and a drop-in OpenAI-compatible API.\n\n![Nexus use cases](assets\u002Fuse-cases.gif)\n\n## Why Nexus\n\nMost teams start with one LLM endpoint. That works until traffic grows, latency starts swinging, finance asks why every request hits the most expensive model, and incident review asks why the app kept calling a degraded provider. Nexus gives the application one stable OpenAI-compatible API while moving model choice, fallback, budget, audit, and routing rationale into infra-owned middleware.\n\nNexus is designed for AI infrastructure engineers running multi-model production pipelines where quality, latency, and cost must be optimized at the same time.\n\n## Problems It Solves\n\n- Issue: every prompt is sent to the same frontier model.\n  Nexus solves this by classifying prompt complexity and routing simple tasks to cheaper low-latency models while reserving premium models for hard prompts.\n\n- Issue: spend grows faster than product usage.\n  Nexus solves this with cost-aware routing, model cost estimates, per-user budget guardrails, and Prometheus cost metrics.\n\n- Issue: code, medical, legal, and general prompts need different quality defaults.\n  Nexus solves this by extracting a domain tag and applying deterministic policy rules such as medical\u002Flegal to Claude Sonnet 4.6 and complex code to GPT-5.5.\n\n- Issue: one provider has an incident and the app fails hard.\n  Nexus solves this with per-provider circuit breakers and automatic fallback chains.\n\n- Issue: provider latency spikes during peak traffic.\n  Nexus solves this with latency-aware routing that tracks rolling p95 latency and penalizes slow providers.\n\n- Issue: teams want to compare models without rewriting product code.\n  Nexus solves this with stable request-id A\u002FB routing selected by the `X-Router-Strategy` header.\n\n- Issue: support and compliance teams ask why a model answered a request.\n  Nexus solves this by persisting durable audit records with `request_id`, selected model, strategy, rationale, latency, token usage, and cost.\n\n- Issue: a single API key can overwhelm the router.\n  Nexus solves this with a token-bucket rate limiter keyed by API key identifier.\n\n- Issue: session or tenant budgets need hard enforcement.\n  Nexus solves this by rejecting requests before dispatch when estimated spend would exceed the configured cap.\n\n- Issue: PII can leak into third-party providers.\n  Nexus solves this with optional regex redaction and a Presidio extension path before provider dispatch.\n\n- Issue: teams need OpenAI compatibility without giving up provider choice.\n  Nexus solves this by exposing `\u002Fv1\u002Fchat\u002Fcompletions` while normalizing OpenAI, Anthropic, Gemini, and Moonshot adapters behind one interface.\n\n- Issue: model routing becomes a hidden product decision.\n  Nexus solves this by making routing policy explicit, testable, observable, and owned in infra.\n\n## Demo Gallery\n\nTerminal routing demo with JSON rationale logs:\n\n![Nexus terminal demo](assets\u002Fdemo.gif)\n\nObserve -> Decide -> Act state-machine demo:\n\n![Nexus decision flow](assets\u002Fdecision-flow.gif)\n\n## Features\n\n- **Router engine** with configurable strategies\n- **Adapter pipeline** with full observability\n- **Async-first** design using `asyncio` + `httpx`\n- **Type-safe** with full `mypy` compliance\n- **Production-ready** with Docker, CI\u002FCD, and structured logging\n\n## Quick Start\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FFrancis1998\u002Fnexus-llm-router.git\ncd nexus-llm-router\npip install -e \".[dev]\"\ncp .env.example .env\nPYTHONPATH=src uvicorn api.main:app --reload\n```\n\n## Quality Gates\n\n```bash\nruff check src\u002F tests\u002F scripts\u002F\nmypy src\u002F\npytest tests\u002F -v\n```\n\n## Docker Compose\n\n```bash\ndocker compose up --build\n```\n\nServices:\n\n- Router: `http:\u002F\u002Flocalhost:8000`\n- Prometheus: `http:\u002F\u002Flocalhost:9090`\n- Grafana: `http:\u002F\u002Flocalhost:3000`\n\n## Routing Strategies\n\nSelect a strategy with `X-Router-Strategy`:\n\n- `rule-based`: domain and complexity priority matrix\n- `classifier`: logistic-regression-style complexity and domain features\n- `cost-optimal`: minimizes estimated cost subject to quality floor\n- `latency-aware`: penalizes providers with poor rolling p95 latency\n- `reliability-aware`: routes to the highest-quality model whose provider circuit is closed, and orders the fallback chain healthy-providers-first\n- `weighted-blend`: selects the model with the highest tunable composite of normalized quality, cost, and latency (weights via `NEXUS_BLEND_*`)\n- `budget-aware`: selects the highest-quality model whose estimated per-request cost stays within a hard ceiling (`NEXUS_REQUEST_COST_CEILING_USD`); the dual of `cost-optimal`\n- `sticky-session`: consistent-hashes `session_id` onto one domain-eligible model, so every turn in a session routes to the same model (context\u002Fprompt-cache affinity) while distinct sessions spread across the pool\n- `value`: selects the model with the best quality-per-dollar ratio, maximizing spend efficiency with no threshold to tune\n- `ab`: deterministic request-id buckets across two model arms\n\n## Documentation\n\n| Document | Description |\n|----------|-------------|\n| [Architecture](ARCHITECTURE.md) | System design and component overview |\n| [Configuration](CONFIGURATION.md) | All configuration options |\n| [Quickstart](QUICKSTART.md) | Local setup and first request |\n| [Safety](SAFETY.md) | Guardrails, fallback, and PII controls |\n| [Contributing](CONTRIBUTING.md) | Development workflow and PR process |\n| [Security](SECURITY.md) | Vulnerability reporting policy |\n| [Changelog](CHANGELOG.md) | Version history |\n\n## License\n\nApache-2.0 © [Francis1998](https:\u002F\u002Fgithub.com\u002FFrancis1998)\n\n*Last updated: 2026-06-26*\n","Nexus-LLM-Router 是一个生产就绪的多大语言模型（LLM）智能路由网关，提供任务感知的模型选择、成本优化、故障自动降级与安全审计能力。它通过 OpenAI 兼容 API（\u002Fv1\u002Fchat\u002Fcompletions）对外暴露统一接口，内部基于提示复杂度、领域标签、实时延迟、模型成本、预算阈值等维度动态路由至 GPT、Claude、Gemini 等后端模型，并支持红队脱敏、令牌桶限流、Prometheus 成本监控与持久化审计日志。适用于需在质量、延迟与成本间协同优化的 AI 基础设施场景，尤其适合已部署多模型且面临预算管控、SLA 保障与合规审计需求的 MLOps 团队。",2,"2026-07-09 02:30:14","CREATED_QUERY"]