[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-1708":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":17,"stars30d":17,"stars90d":16,"forks30d":16,"starsTrendScore":18,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":10,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":24,"readmeContent":25,"aiSummary":26,"trendingCount":16,"starSnapshotCount":16,"syncStatus":15,"lastSyncTime":27,"discoverSource":28},1708,"flow-llm-router","flowapi-net\u002Fflow-llm-router","flowapi-net","The Ultimate \"Token Saver\"","https:\u002F\u002Fflowapi.net\u002F",null,"Python",70,21,6,2,0,1,3,4.03,false,"main",true,[],"2026-06-12 02:00:31","\u003Cdiv align=\"center\">\n  \u003Cimg src=\"flowapi_logo.png\" alt=\"FlowAPI\" width=\"500\">\n  \u003Ch1>🚀 Flow-LLM-Router: The Ultimate \"Token Saver\"\u003C\u002Fh1>\n  \u003Cp>\n      \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fflow-llm-router\u002F\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fv\u002Fflow-llm-router\" alt=\"PyPI\">\u003C\u002Fa>\n    \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpython-≥3.11-blue\" alt=\"Python\">\n    \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-green\" alt=\"License\">\n  \u003C\u002Fp>\n\u003C\u002Fdiv>\n\n\n### 👉  To drastically reduce your AI Agent costs, you need to optimize both Token Price and Token Usage. \n\n### We provide the complete 2-step solution:\n\n- ###  ⚡️ The Most Cost-Effective LLM Platformn at [FlowAPI Platform](https:\u002F\u002Fflowapi.net).\n\n- ###   🐈 **Flow-LLM-Router** Your Local AI Control Plane to Save Token.\n***\n\n## 🔥 Core Features of the flow-llm-router\n\n- ###  📊 Automated Token Analytics Dashboard\n\n- ###  🧠 Smart API Routing (Rules & Classifiers)\n\n- ###  🧰 Dynamic Skill Loading (Eliminate the \"Token Tax\")\n\n- ###  🔒 Secure, Local API Key Management\n\n***\n\n## At A Glance\n\n- [The Problem](#the-problem)\n- [Why Flow LLM Router](#why-flow_llm_router)\n- [Who It Is For](#who-it-is-for)\n- [Use Cases](#use-cases)\n- [Why Pair It With FlowAPI](#why-pair-it-with-flowapi)\n- [Key Capabilities](#key-capabilities)\n- [Quick Start](#quick-start)\n- [Documentation](#documentation)\n- [FAQ](#faq)\n\n## Why Flow-LLM-Router\n\n- **OpenAI-compatible by default**: point existing SDKs to `http:\u002F\u002Fhost:7798\u002Fv1` and keep most client code unchanged.\n- **Local-first observability**: request logs, token usage, latency, and routing metadata stay in local SQLite.\n- **Encrypted provider vault**: provider API keys are stored encrypted at rest and unlocked only when needed.\n- **Multi-provider routing**: route across OpenAI, Anthropic, Gemini, DeepSeek, Qwen, Groq, custom OpenAI-compatible endpoints, and more.\n- **Operator-friendly dashboard**: manage providers, models, caller tokens, logs, analytics, router settings, and integration snippets from one UI.\n- **Optional smart router**: use fast rule-based complexity scoring or RouteLLM-powered classifier routing.\n\n## Who It Is For\n\n- teams building AI agents, copilots, or workflow automation on top of OpenAI-style SDKs\n- developers who want local logs and routing visibility without adopting a hosted observability layer\n- operators who need one stable endpoint in front of multiple model providers\n- builders who want to combine lower-cost upstream pricing with local request optimization\n\n## Use Cases\n\n- **AI agents and copilots**: centralize routing and provider auth behind one OpenAI-compatible endpoint\n- **Multi-model workflows**: send easy tasks to cheap models and reserve premium models for high-value reasoning\n- **Private internal tooling**: keep prompts, logs, and credentials inside your own environment\n- **Cost debugging**: identify which models, prompts, and traffic patterns are silently increasing your bill\n- **Gateway standardization**: give multiple applications one stable base URL even when upstream providers differ\n\n## What It Ships\n\n| Area          | What Flow LLM Router provides                                                                                                            |\n| ------------- | --------------------------------------------------------------------------------------------------------------------------------- |\n| **Proxy**     | `POST \u002Fv1\u002Fchat\u002Fcompletions`, streaming chat completions, `POST \u002Fv1\u002Fembeddings`, and `GET \u002Fv1\u002Fmodels`.                             |\n| **Dashboard** | Analytics, request logs, provider management, model catalog, router configuration, caller token management, and integration help. |\n| **Security**  | Encrypted provider key storage, master-password unlock flow, caller token access control, IP allowlisting, and log redaction.     |\n| **Routing**   | Pass-through mode, rule-based complexity routing, and optional RouteLLM classifier routing with graceful fallback.                |\n| **Catalog**   | Sync model lists from provider `\u002Fmodels` endpoints and reuse them in the UI and router configuration.                             |\n| **Packaging** | Python package, FastAPI app, Typer CLI, and a statically exported Next.js dashboard served by the API process.                    |\n\n## Architecture\n\n```text\nYour App \u002F Agent \u002F SDK\n        |\n        | OpenAI-compatible requests\n        v\n  Flow LLM Router Proxy  ------------------------------+\n        |                                       |\n        | auth + routing + logging              |\n        v                                       |\n  LiteLLM forwarding layer                      |\n        |                                       |\n        +--> Provider vault (encrypted keys)    |\n        +--> Smart router service               |\n        +--> SQLite logs + model catalog        |\n        +--> Static dashboard UI                |\n        |\n        v\nOpenAI \u002F Anthropic \u002F Gemini \u002F DeepSeek \u002F Qwen \u002F custom OpenAI-compatible backends\n```\n\n## Why Not Direct SDK Calls\n\n| Approach                              | What you miss                                                                                                             |\n| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |\n| **Call OpenAI or Anthropic directly** | No unified routing, no local gateway, no provider abstraction, no centralized logs                                        |\n| **Use a generic proxy only**          | Basic forwarding is not enough if you also want routing policy, encrypted key management, and operator-friendly analytics |\n| **Use Flow LLM Router**                      | Keep one OpenAI-compatible endpoint while adding routing, logs, local security controls, and provider portability         |\n\n## Why Pair It With FlowAPI\n\nFlow LLM Router and FlowAPI solve different layers of the cost stack:\n\n| Layer                      | What optimizes it                                               |\n| -------------------------- | --------------------------------------------------------------- |\n| **Token unit price**       | [FlowAPI.net](https:\u002F\u002Fflowapi.net) as the upstream endpoint     |\n| **Token usage efficiency** | Flow LLM Router's routing, request visibility, and local control plane |\n\nIf you are serious about cost control, you usually want both.\n\n## Where The Savings Come From\n\nFlow LLM Router is not magic. It improves cost structure through a few concrete levers:\n\n| Cost lever                       | What Flow LLM Router changes                                                | Why it matters                                                          |\n| -------------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------- |\n| **Model selection**              | Routes lower-complexity work to cheaper models                       | Prevents expensive models from handling trivial tasks                   |\n| **Operational visibility**       | Exposes local logs, model usage, latency, and routing data           | Makes waste visible so teams can actually fix it                        |\n| **Provider abstraction**         | Lets you point one app surface at different upstream providers       | Makes it easier to optimize for economics without rewriting client code |\n| **Prompt discipline foundation** | Adds a local control layer for future prompt and skills optimization | Reduces the chance that token inefficiency stays hidden in agent stacks |\n\n## Key Capabilities\n\n### 1. Drop-in OpenAI compatibility\n\nFlow LLM Router exposes an OpenAI-style API surface so existing clients can usually switch by changing only the base URL and token source.\n\nSupported endpoints today:\n\n- `POST \u002Fv1\u002Fchat\u002Fcompletions`\n- `POST \u002Fv1\u002Fembeddings`\n- `GET \u002Fv1\u002Fmodels`\n\nStreaming chat completions are forwarded as SSE.\n\n### 2. Local observability that is actually usable\n\nEvery proxied request can be logged with:\n\n- requested model and routed model\n- provider\n- prompt, response, and error status\n- token usage\n- latency\n- smart-router score and tier\n- session metadata\n\nThe dashboard surfaces this through overview metrics, timelines, provider\u002Fmodel breakdowns, and per-request log detail views.\n\n### 3. Built-in security controls\n\nFlow LLM Router separates **caller access** from **provider credentials**:\n\n- **Caller tokens** control who is allowed to use the proxy.\n- **Provider keys** are stored encrypted in SQLite.\n- **Master-password unlock** protects the provider vault.\n- **IP allowlisting** limits where requests may come from.\n- **Log redaction** helps prevent accidental credential leakage in persisted logs.\n\nIf no caller tokens exist yet, proxy access remains open for easier local setup. Once tokens are created, valid caller tokens become mandatory.\n\n### 4. Smart routing without leaving the box\n\nFlow LLM Router supports three routing modes:\n\n- **`off`**: pass through the requested model unchanged\n- **`complexity`**: local rule-based routing using a 7-dimension prompt complexity scorer\n- **`classifier`**: RouteLLM-based routing, mapped back into Flow LLM Router's four-tier model layout\n\nBoth routing strategies use the same tier mapping:\n\n- `SIMPLE`\n- `MEDIUM`\n- `COMPLEX`\n- `REASONING`\n\nIf RouteLLM is unavailable or fails at runtime, Flow LLM Router falls back to rule-based routing instead of breaking requests.\n\n### 5. Analytics for finding your token assassins\n\nFlow LLM Router includes a built-in local dashboard so you can inspect:\n\n- which models are used most often\n- which providers consume the most tokens\n- which requests are slow, error-prone, or over-routed\n- how routing tiers are distributed over time\n\nCost optimization gets much easier once the waste is visible.\n\n### 6. Skills-ready foundation\n\nFlow LLM Router includes optional skills-related configuration and package extras for teams exploring retrieval- and tool-oriented prompt optimization.\n\nIn the current repository, this is best understood as a foundation for prompt-efficiency work rather than a fully productized dynamic skill-routing system.\n\n## Install\n\n### Base install\n\n```bash\npip install flow-llm-router\n```\n\n### Optional extras\n\n```bash\n# RouteLLM-based classifier routing\npip install 'flow-llm-router[classifier]'\n\n# ChromaDB-backed skills support\npip install 'flow-llm-router[skills]'\n```\n\n## Quick Start\n\n### 1. Start Flow LLM Router\n\n```bash\nflow-router start\n```\n\nDefault endpoints:\n\n- Dashboard: `http:\u002F\u002F127.0.0.1:7798`\n- Proxy base URL: `http:\u002F\u002F127.0.0.1:7798\u002Fv1`\n- OpenAPI docs: `http:\u002F\u002F127.0.0.1:7798\u002Fdocs`\n\nTo load a custom config file:\n\n```bash\nexport FLOWGATE_CONFIG=\u002Fpath\u002Fto\u002Fflow_llm_router.yaml\nflow-router start\n```\n\n### 2. Open the dashboard\n\nGo to \u003Chttp:\u002F\u002F127.0.0.1:7798> and:\n\n- set up or unlock the vault\n- add one or more provider keys\n- optionally sync provider models\n- optionally create caller tokens\n- optionally configure the smart router\n\n### 3. Point your SDK at Flow LLM Router\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http:\u002F\u002F127.0.0.1:7798\u002Fv1\",\n    api_key=\"fgt_your_caller_token_or_dummy\",\n)\n\nresponse = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello from Flow LLM Router\"}],\n)\n\nprint(response.choices[0].message.content)\n```\n\nIf you have not created any caller tokens yet, Flow LLM Router accepts requests without token enforcement. In production, create caller tokens and restrict access explicitly.\n\n## Configuration\n\nCopy the example file and adjust it for your environment:\n\n```bash\ncp flow_llm_router.yaml.example flow_llm_router.yaml\n```\n\nMain configuration areas:\n\n- `server`: bind host and port\n- `smart_router`: routing strategy and tier mappings\n- `skills`: optional retrieval support\n- `database`: SQLite database path\n- `logging`: prompt\u002Fresponse logging and secret redaction\n- `security`: vault, auth token TTL, persisted master key path, and IP allowlisting\n\nExample smart-router configuration:\n\n```yaml\nsmart_router:\n  enabled: true\n  strategy: complexity   # complexity | classifier | off\n  tiers:\n    SIMPLE: gpt-4o-mini\n    MEDIUM: gpt-4o\n    COMPLEX: claude-sonnet\n    REASONING: o1-preview\n```\n\nEnvironment references such as `${OPENAI_API_KEY}` are supported in YAML values.\n\n## CLI\n\n| Command               | Purpose                                                 |\n| --------------------- | ------------------------------------------------------- |\n| `flow-router start`   | Start the FastAPI server and static dashboard           |\n| `flow-router add-key` | Interactively add a provider key to the encrypted vault |\n| `flow-router version` | Print the installed Flow LLM Router version                    |\n\nUse `flow-router --help` for all flags and options.\n\n## Development\n\n```bash\ngit clone \u003Cyour-repo-url>\ncd flow-llm-router\npython -m venv .venv\nsource .venv\u002Fbin\u002Factivate\npip install -e '.[dev,classifier]'\npytest -q\n```\n\nFrontend workflow:\n\n```bash\ncd frontend\nnpm ci\nnpm run build\n```\n\nTo build the static dashboard and copy it into the package:\n\n```bash\nbash scripts\u002Fbuild_frontend.sh\n```\n\n## Documentation\n\n| Document                                                   | Focus                                                                       |\n| ---------------------------------------------------------- | --------------------------------------------------------------------------- |\n| [docs\u002FDESIGN.md](docs\u002FDESIGN.md)                           | Design entry point and guide to the current documentation structure         |\n| [docs\u002FARCHITECTURE.md](docs\u002FARCHITECTURE.md)               | System layout, runtime components, request flow, and storage model          |\n| [docs\u002FAPI\\_AND\\_OPERATIONS.md](docs\u002FAPI_AND_OPERATIONS.md) | API surface, auth flow, provider onboarding, and operational notes          |\n| [docs\u002FSMART\\_ROUTER.md](docs\u002FSMART_ROUTER.md)              | Smart-router strategies, config schema, API payloads, and fallback behavior |\n| [docs\u002FDEVELOPMENT.md](docs\u002FDEVELOPMENT.md)                 | Local development, frontend build flow, testing, and contribution notes     |\n| [docs\u002FTESTING.md](docs\u002FTESTING.md)                         | Backend test coverage, smoke-test entry points, and verification guidance   |\n\n## FAQ\n\n### Is Flow LLM Router a hosted proxy?\n\nNo. This repository is the local-first gateway layer you run yourself.\n\n### Does Flow LLM Router replace LiteLLM?\n\nNo. Flow LLM Router uses LiteLLM as the forwarding layer and adds local routing, security, and observability on top.\n\n### Does Flow LLM Router require me to rewrite my OpenAI SDK integration?\n\nUsually no. In most cases you only change the base URL and, if enabled, the caller token.\n\n### Can I use Flow LLM Router without FlowAPI.net?\n\nYes. Flow LLM Router works independently with direct provider APIs and custom OpenAI-compatible endpoints. FlowAPI is an optional upstream pairing for better token pricing.\n\n### Does Flow LLM Router already implement full dynamic top-k skill injection?\n\nNot as a finished production feature in the current repository. The project includes skills-related configuration and extension hooks, but the README positions this today as an optimization direction and foundation rather than a fully shipped headline workflow.\n\n### Where do my logs and keys live?\n\nRequest metadata and catalog data live in local SQLite. Provider API keys are stored encrypted and unlocked only when needed.\n\n## Project Status\n\nFlow LLM Router is currently **alpha** and focused on shipping a tight local gateway experience for AI developers and small teams.\n\nCurrent strengths:\n\n- OpenAI-compatible local proxying\n- encrypted provider credential handling\n- integrated analytics and logs\n- configurable smart routing\n- model catalog sync\n- local dashboard operations\n\nCurrent gaps:\n\n- no formal multi-node deployment story yet\n- no official container or Helm distribution in this repository yet\n- no guaranteed backward-compatibility promise across early releases\n\n## Roadmap\n\n- [x] OpenAI-compatible local proxy\n- [x] Encrypted provider vault\n- [x] Model catalog sync\n- [x] Smart router with rule-based and classifier modes\n- [x] Local analytics dashboard\n- [ ] Richer cost attribution across agents and workflows\n- [ ] Better router evaluation and decision explainability\n- [ ] Production-ready packaging and deployment examples\n- [ ] Stronger admin auth and multi-user operations\n- [ ] More polished benchmarking, demos, and screenshots\n\n## Roadmap Direction\n\nThe current codebase suggests a clear next path for the project:\n\n- broaden provider ergonomics for custom OpenAI-compatible gateways\n- deepen router evaluation and cost\u002Fquality observability\n- improve deployment and packaging workflows\n- harden authentication and admin-session handling\n- expand documentation and operator guidance\n\n## Contributing\n\nPull requests are welcome.\n\nGood contribution areas:\n\n- provider integrations and OpenAI-compatible endpoint handling\n- routing strategy improvements and classifier evaluation\n- analytics and dashboard polish\n- packaging, deployment, and operator workflows\n- documentation, examples, and benchmark material\n\nIf you are extending behavior, keep the docs aligned with the implementation and prefer changes that remain inspectable and local-first.\n\n## Star History\n\n[![Star History Chart](https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=flowapi-net\u002Fflow-llm-router\\&type=Date)](https:\u002F\u002Fwww.star-history.com\u002F#flowapi-net\u002Fflow-llm-router\\&Date)\n\n## License\n\nMIT\n","Flow-LLM-Router 是一个旨在大幅降低AI代理成本的“Token Saver”工具。其核心功能包括自动化的Token分析仪表板、智能API路由（基于规则和分类器）、动态技能加载以消除“Token税”，以及安全的本地API密钥管理。该项目支持多提供商路由，兼容OpenAI及其他多个主流AI平台，并提供了一个操作友好的控制面板来管理提供商、模型、日志等。适用于构建AI代理、协同工作流程自动化、需要本地日志与路由可见性的开发者，以及希望在多个模型提供商前设置单一稳定端点的操作员。","2026-06-11 02:45:34","CREATED_QUERY"]