[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-95113":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":9,"totalLinesOfCode":9,"stars":12,"forks":13,"watchers":14,"openIssues":14,"contributorsCount":9,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":14,"stars30d":14,"stars90d":14,"forks30d":14,"starsTrendScore":14,"compositeScore":15,"rankGlobal":9,"rankLanguage":9,"license":9,"archived":16,"fork":16,"defaultBranch":17,"hasWiki":16,"hasPages":16,"topics":9,"createdAt":9,"pushedAt":9,"updatedAt":18,"readmeContent":19,"aiSummary":20,"trendingCount":14,"starSnapshotCount":14,"syncStatus":21,"lastSyncTime":9,"discoverSource":22},95113,"ODS","Osmantic\u002FODS","Osmantic","Turn your PC, Mac, or Linux box into an AI server. LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.",null,"https:\u002F\u002Fgithub.com\u002FOsmantic\u002FODS","Python",4553,685,0,57.51,false,"main","2026-08-24 04:01:23","\u003Cdiv align=\"center\">\n\n# ODS\n\n**Osmantic Deployment System**\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fosmantic.com\" target=\"_blank\" rel=\"noopener noreferrer\">\n    \u003Cimg src=\"ods\u002Fdocs\u002Fimages\u002Fosmantic-lockup.png\" alt=\"Osmantic\" width=\"800\">\n  \u003C\u002Fa>\n\u003C\u002Fp>\n\n**Turn your PC, Mac, or Linux box into a private AI server.**\n\nAI server and homelab setup is rapidly becoming a solved problem.\nIt should feel that way for everyone.\n\n[![License: Apache 2.0](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-Apache%202.0-blue.svg)](LICENSE)\n[![GitHub Stars](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002FOsmantic\u002FODS)](https:\u002F\u002Fgithub.com\u002FOsmantic\u002FODS\u002Fstargazers)\n[![Release](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fv\u002Frelease\u002FOsmantic\u002FODS)](https:\u002F\u002Fgithub.com\u002FOsmantic\u002FODS\u002Freleases)\n\n[![Watch the demo](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDemo-Watch%20on%20YouTube-red?logo=youtube)](https:\u002F\u002Fyoutu.be\u002FnO8xFNHX-HA)\n\n\u003C\u002Fdiv>\n\nODS installs and wires together everything you need to run AI locally, so you do not have to assemble Ollama, Open WebUI, n8n, ComfyUI, and privacy tools by hand:\n\n- **Local model inference** — run open models on your own hardware\n- **ChatGPT-style web UI** — talk to your models from any browser\n- **Control dashboard** — manage models, services, setup, GPU status, and extensions from one place\n- **Voice, agents, and workflows** — build automations that can listen, speak, call tools, and get work done\n- **RAG and search** — connect local documents, private search, and retrieval workflows\n- **Image generation** — run local image tools without sending prompts to a hosted API\n- **Privacy and ops** — keep service auth, secrets, observability, and diagnostics in one local stack\n\nNo cloud required. No subscriptions required. Your prompts and data stay on your machine unless you choose otherwise. Cloud and hybrid API modes are optional when you want them.\n\n**Release validation:** Operational changes are checked with a release-grade\nfleet and distro lab: zero-prereq bootstrap, fresh installs, product flows,\nfull-model capabilities, lifecycle recovery, and the final User Green gate. See\n[Release Validation](ods\u002Fdocs\u002FRELEASE_VALIDATION.md) for what a green\nrun proves.\n\n**Repo layout:** the repository root holds the public README, installers,\nsecurity policy, GitHub workflows, and project coordination docs. The\n`ods\u002F` directory is the product runtime: services, installer phases,\ncompose overlays, dashboard, CLI, tests, and operator docs.\n\n**Stable consumption:** `v2.6.0` is the current stable release. `main` moves\nquickly; use it for active development and validation candidates. For forks,\nappliances, labs, or production-like installs, pin a tagged release or audited\ncommit and keep your own validation receipt. Stable patch fixes land on\n`release\u002F2.6.x` before being merged forward. See\n[Release Channels](ods\u002Fdocs\u002FRELEASE_CHANNELS.md),\n[Installer Trust](ods\u002Fdocs\u002FINSTALLER_TRUST.md), and\n[Forkability](ods\u002Fdocs\u002FFORKABILITY.md).\n\n## Get Started\n\nChoose your system, copy the block, run it in a normal terminal. ODS installs the stack, picks a model for your hardware, starts the services, and gives you the local web UI.\n\n**Linux or macOS**\n\n```bash\ncurl -fsSL https:\u002F\u002Finstall.osmantic.com\u002Fods.sh | bash\n```\n\n**Windows PowerShell**\n\n```powershell\n$ProgressPreference = \"SilentlyContinue\"\n$odsSrc = Join-Path $env:TEMP (\"ods-install-\" + [guid]::NewGuid().ToString(\"N\"))\n$odsZip = Join-Path $odsSrc \"ods-main.zip\"\nNew-Item -ItemType Directory -Path $odsSrc | Out-Null\nInvoke-WebRequest \"https:\u002F\u002Fgithub.com\u002FOsmantic\u002FODS\u002Farchive\u002Frefs\u002Fheads\u002Fmain.zip\" -OutFile $odsZip\nExpand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force\ncd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName\nSet-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass\n.\\install.ps1\n```\n\nPrerequisites: Docker must be installed and running. On Windows, use Docker Desktop with the WSL2 backend enabled and run the block in a normal, non-Administrator PowerShell window.\n\nThe hosted Linux\u002FmacOS endpoint proxies the current bootstrap from repository `main`.\nReviewed merges reach it automatically after edge-cache refresh. `ODS_REF` selects a compatible repository checkout. See\n[Installer Trust](ods\u002Fdocs\u002FINSTALLER_TRUST.md) to inspect the script or install\na stable release or audited commit manually.\n\nWindows users should not run the `curl ... | bash` command from PowerShell. The PowerShell block above downloads the source ZIP and runs the same Windows installer used by the clone-based workflow. For more detail, see the [Windows Quickstart](ods\u002Fdocs\u002FWINDOWS-QUICKSTART.md).\n\nAfter install, open **http:\u002F\u002Flocalhost:3000** and start chatting.\n\nUninstall later with the matching platform command:\n\n```bash\ncd ~\u002Fods\n.\u002Fods-uninstall.sh --force\n```\n\n```powershell\n$installDir = \"$env:USERPROFILE\\ods\"\ncd $installDir\n.\\ods.ps1 uninstall --force\n```\n\nWindows recovery note: if the runtime folder is partial and `.\\ods.ps1` is missing, run the same command from a source checkout as `.\\ods\\installers\\windows\\ods.ps1 uninstall --force`. It removes Docker resources labelled as the ODS compose project before removing the runtime directory.\n\n> **API endpoint:** Linux Docker installs expose llama-server on **http:\u002F\u002Flocalhost:11434** by default (`OLLAMA_PORT`) while containers use `llama-server:8080`. macOS native Metal and Windows native\u002FLemonade paths use **http:\u002F\u002Flocalhost:8080** unless overridden. Open WebUI stays on **http:\u002F\u002Flocalhost:3000**.\n\n> **No GPU?** ODS also runs in cloud mode — same full stack, powered by OpenAI\u002FAnthropic\u002FTogether APIs instead of local inference:\n> ```bash\n> .\u002Finstall.sh --cloud\n> ```\n\n> **Port conflicts?** Every port is configurable via environment variables. See [`.env.example`](ods\u002F.env.example) for the full list, or override at install time:\n> ```bash\n> WEBUI_PORT=9090 .\u002Finstall.sh\n> ```\n\n**New here?** Read the [Friendly Guide](ods\u002Fdocs\u002FHOW-ODS-SERVER-WORKS.md) or [listen to the audio version](https:\u002F\u002Fopen.spotify.com\u002Fepisode\u002F40MvqJ41bC8cEgvUyOyE3K) — a complete walkthrough of what ODS is, how it works, and how to make it your own. No technical background needed.\n\n---\n\n## At A Glance\n\n| Question | Answer |\n|----------|--------|\n| **What is it?** | A local AI server stack for your own hardware, with a one-command Linux\u002FmacOS installer and a PowerShell installer for Windows. |\n| **Who is it for?** | People who want private AI at home, in a lab, or on a workstation without hand-wiring a dozen services. |\n| **What do I get?** | Local inference, Open WebUI chat, a control dashboard, voice, agents, workflows, RAG, search, image generation, privacy tools, observability, and developer tools. |\n| **What does it run on?** | Linux, Windows with WSL2\u002FDocker Desktop, and macOS Apple Silicon. |\n| **Is cloud required?** | No. Local mode is the default; cloud and hybrid API modes are optional. |\n\n| If you know... | ODS adds... |\n|----------------|----------------------|\n| **Ollama \u002F llama.cpp** | The surrounding server stack: chat, dashboard, voice, RAG, workflows, agents, privacy, and service management. |\n| **Open WebUI** | A full installer and control plane around Open WebUI, plus pre-wired local services. |\n| **AnythingLLM** | Broader local AI appliance behavior beyond RAG: inference, chat, voice, workflows, image generation, and ops. |\n| **n8n self-hosted AI starter kits** | Workflow automation as one part of a larger private AI server. |\n\n---\n\n> **Current Platform Support**\n>\n> | Platform | Status |\n> |----------|--------|\n> | **Linux** (NVIDIA + AMD + Intel Arc) | **Supported** — install and run today |\n> | **Windows** (NVIDIA + AMD) | **Supported** — install and run today |\n> | **macOS** (Apple Silicon) | **Supported** — install and run today |\n>\n> **Tested Linux distros:** Ubuntu 24.04\u002F22.04, Debian 12, Linux Mint 21.3, Fedora 41+, Rocky Linux 9, Arch Linux, Manjaro, CachyOS, and openSUSE Tumbleweed. Other distros using apt, dnf, pacman, or zypper should also work — [open an issue](https:\u002F\u002Fgithub.com\u002FOsmantic\u002FODS\u002Fissues) if yours doesn't.\n>\n> **Release validation:** Operational changes run through a release-grade gate\n> that covers zero-prereq bootstrap, clean installs, product behavior,\n> full-model capabilities, lifecycle recovery, and User Green. See\n> [Release Validation](ods\u002Fdocs\u002FRELEASE_VALIDATION.md) and the\n> [Validation Matrix](ods\u002Fdocs\u002FVALIDATION-MATRIX.md).\n>\n> **Windows:** Requires Docker Desktop with WSL2 backend. NVIDIA GPUs use Docker GPU passthrough; AMD Strix Halo runs through the platform-specific accelerated path documented in the Windows installer and support matrix.\n>\n> **macOS:** Requires Apple Silicon (M1+) and Docker Desktop. llama-server runs natively with Metal GPU acceleration; all other services run in Docker.\n>\n> See the [Support Matrix](ods\u002Fdocs\u002FSUPPORT-MATRIX.md) for supported\n> platform claims and the [Validation Matrix](ods\u002Fdocs\u002FVALIDATION-MATRIX.md)\n> for the layered test surface used to test those claims.\n\n---\n\n## Why ODS?\n\nA handful of companies control the vast majority of global AI traffic — and with it, your data, your costs, and your uptime. Every query you send to a centralized provider is business intelligence you don’t own, running on infrastructure you don’t control, priced on terms you can’t negotiate.\n\nIf AI is becoming critical infrastructure, it shouldn’t be rented. Self-hosting local AI should be a sovereign human right, not a career choice.\n\nBecause running your own AI shouldn't require a CS degree and a weekend of debugging CUDA drivers. Right now, setting up local AI means stitching together a dozen projects, writing Docker configs from scratch, and praying everything talks to each other. Most people give up and go back to paying OpenAI.\n\nWe built ODS so you don't have to.\n\n- **One command** — detects your GPU, picks the right model, generates credentials, launches everything\n- **Chatting in under 2 minutes** — bootstrap mode gives you a working model instantly while your full model downloads in the background\n- **Full service stack, pre-wired** — chat, agents, voice, workflows, search, RAG, image generation, privacy tools, observability, and developer tools. All talking to each other out of the box\n- **Fully moddable** — every service is an extension. Drop in a folder, run `ods enable`, done\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>Manual install (Linux)\u003C\u002Fb>\u003C\u002Fsummary>\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FOsmantic\u002FODS.git\ncd ODS\u002Fods\n.\u002Finstall.sh\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>Windows (PowerShell)\u003C\u002Fb>\u003C\u002Fsummary>\n\nRequires [Docker Desktop](https:\u002F\u002Fwww.docker.com\u002Fproducts\u002Fdocker-desktop\u002F) with WSL2 backend enabled.\n**Install Docker Desktop first and make sure it is running before you start.**\n\nOpen a normal **PowerShell** session and run:\n\n```powershell\n$ProgressPreference = \"SilentlyContinue\"\n$odsSrc = Join-Path $env:TEMP (\"ods-install-\" + [guid]::NewGuid().ToString(\"N\"))\n$odsZip = Join-Path $odsSrc \"ods-main.zip\"\nNew-Item -ItemType Directory -Path $odsSrc | Out-Null\nInvoke-WebRequest \"https:\u002F\u002Fgithub.com\u002FOsmantic\u002FODS\u002Farchive\u002Frefs\u002Fheads\u002Fmain.zip\" -OutFile $odsZip\nExpand-Archive -LiteralPath $odsZip -DestinationPath $odsSrc -Force\ncd (Get-ChildItem -LiteralPath $odsSrc -Directory | Select-Object -First 1).FullName\nSet-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass\n.\\install.ps1\n```\n\n> The `Set-ExecutionPolicy` command allows the installer script to run in the current session. It does not change your system-wide policy.\n> Running as Administrator is not recommended for the installer because user-level paths such as `.opencode`, `data\u002F`, and `.env` can be created with admin-owned permissions.\n\nThe installer detects your GPU, picks the right model, generates credentials, starts all services, and creates a Desktop shortcut to the Dashboard. Manage from the runtime directory with `.\\ods.ps1 status`; uninstall with `.\\ods.ps1 uninstall --force`.\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>macOS (Apple Silicon)\u003C\u002Fb>\u003C\u002Fsummary>\n\nRequires Apple Silicon (M1+) and [Docker Desktop](https:\u002F\u002Fwww.docker.com\u002Fproducts\u002Fdocker-desktop\u002F).\n**Install Docker Desktop first and make sure it is running before you start.**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FOsmantic\u002FODS.git\ncd ODS\u002Fods\n.\u002Finstall.sh\n```\n\nThe installer detects your chip, picks the right model for your unified memory, launches llama-server natively with Metal acceleration, and starts all other services in Docker. Manage with `.\u002Fods-macos.sh status`.\n\nSee the [macOS Quickstart](ods\u002Fdocs\u002FMACOS-QUICKSTART.md) for details.\n\n\u003C\u002Fdetails>\n\n---\n\n## What's In The Box\n\n### Chat & Inference\n- **Open WebUI** — full-featured chat interface with conversation history, web search, document upload, and [30+ languages](https:\u002F\u002Fdocs.openwebui.com)\n- **llama-server** — high-performance LLM inference with continuous batching, auto-selected for your GPU; Linux Docker host API defaults to `localhost:11434`, native macOS\u002FWindows paths use `localhost:8080`, and container API runs on `8080`\n- **LiteLLM** — API gateway supporting local\u002Fcloud\u002Fhybrid modes\n- **TEI Embeddings** — text embedding service for RAG and search workflows\n\n### Voice\n- **Whisper** — speech-to-text\n- **Kokoro** — text-to-speech\n\n### Agents & Automation\n- **Hermes Agent** — default local-first autonomous\u002Fbrowser agent with memory, skills, and a magic-link-gated proxy\n- **OpenClaw** — deprecated legacy autonomous agent, still opt-in during the migration window\n- **n8n** — workflow automation with 400+ integrations (Slack, email, databases, APIs)\n- **APE** — Agent Policy Engine for auditing and governing autonomous tool calls\n- **OpenCode** — browser-based AI coding assistant wired to the local stack\n- **Memory Shepherd** — host\u002Fsystemd helper for agent memory lifecycle management\n\n### Knowledge & Search\n- **Qdrant** — vector database for retrieval-augmented generation (RAG)\n- **SearXNG** — self-hosted web search (no tracking)\n- **Perplexica** — deep research engine\n- **Brave Search** — optional paid Brave Search API integration\n\n### Creative\n- **ComfyUI** — node-based image generation\n\n### Privacy & Ops\n- **Privacy Shield** — PII scrubbing proxy for API calls\n- **Dashboard** — real-time GPU metrics, service health, model management\n- **Dashboard API** — service health, setup, status, metrics, and management API behind the dashboard\n- **Token Spy** — token usage monitor for local and proxied LLM traffic\n- **Langfuse** — optional LLM observability and tracing\n\n---\n\n## Hardware Auto-Detection\n\nThe installer detects your GPU and first assigns a deterministic hardware tier. Linux and macOS then run the versioned catalog selector (`ods\u002Fscripts\u002Fselect-model.py`), while Windows uses the PowerShell catalog selector in `ods\u002Finstallers\u002Fwindows\u002Flib\u002Ftier-map.ps1`; both read `ods\u002Fconfig\u002Fmodel-library.json` to choose the best installable GGUF for the detected memory envelope. The final choice is written to `.env` as `LLM_MODEL`, `GGUF_FILE`, `MAX_CONTEXT`, and `MODEL_RECOMMENDATION_*`.\n\n`MODEL_PROFILE=qwen` is the default non-Gemma catalog profile, so the effective pick can be Qwen, Phi, or DeepSeek depending on what fits best. `MODEL_PROFILE=gemma4` forces Gemma 4 where available, and `MODEL_PROFILE=auto` uses Gemma 4 on NVIDIA, Apple Silicon, and Intel Arc tiers. Override tier selection with `.\u002Finstall.sh --tier 3`; override the model family with `MODEL_PROFILE=gemma4 .\u002Finstall.sh` or `MODEL_PROFILE=auto .\u002Finstall.sh`.\n\nWhen Hermes is enabled, which is the default agent path, installers keep the first-run bootstrap model at a 64K context floor and promote the full local model context to 128K where the selected model supports it. That avoids Hermes's hard 64K minimum while preserving the under-2-minute first chat experience. The examples below are current catalog-selector outputs for common hardware envelopes; exact installs can differ with detected VRAM\u002FRAM, host architecture, existing downloads, or explicit profile overrides. Throughput still needs a local benchmark after first launch.\n\n### NVIDIA\n\n| Tier \u002F envelope | Current default catalog pick | Context | Example hardware |\n|------|--------------|---------|--------------|\n| 0 \u002F 8 GB CPU fallback | Qwen3.5 2B (Q4_K_M) | 8K | Low-RAM CPU-only |\n| 1 \u002F 8 GB discrete VRAM | Qwen3.5 9B (Q4_K_M) | 32K | RTX 4060, RTX 3060 12GB |\n| 2 \u002F 12 GB discrete VRAM | Phi-4 14B (Q4_K_M) | 16K | RTX 4070-class cards |\n| 3 \u002F 24 GB discrete VRAM | Qwen3.5 27B (Q4_K_M) | 32K | RTX 4090, A6000 |\n| 4 \u002F 48 GB discrete VRAM | DeepSeek R1 Distill Llama 70B (Q4_K_M) | 32K | A6000 Ada, L40S |\n| NV_ULTRA \u002F 90+ GB amd64 discrete VRAM | Qwen3 Coder Next (Q4_K_M) | 128K | Multi-GPU A100\u002FH100 |\n| NV_ULTRA \u002F 90+ GB arm64 unified memory | Qwen3.6 35B-A3B (UD-Q4_K_M) | 128K | DGX Spark \u002F GB10-class hosts |\n\n### AMD Strix Halo (Unified Memory)\n\n| Tier \u002F envelope | Current default catalog pick | Context | Hardware |\n|------|--------------|---------|----------|\n| SH_COMPACT \u002F 64 GB unified RAM | Qwen3.6 35B-A3B (UD-Q4_K_M) | 128K | Ryzen AI MAX+ 395 (64GB) |\n| SH_LARGE \u002F 96 GB unified RAM | DeepSeek R1 Distill Llama 70B (Q4_K_M) | 32K | Ryzen AI MAX+ 395 (96GB) |\n| SH_LARGE \u002F 124 GB unified RAM | Qwen3.6 35B-A3B (UD-Q4_K_M) | 128K | Ryzen AI MAX+ 395 (128GB class) |\n\nThe selector routes unified-memory hosts away from Qwen3 Coder Next when that model would otherwise be selected, because current repo policy documents correctness issues on those backends.\n\n### Apple Silicon (Unified Memory, Metal)\n\n| Tier \u002F envelope | Current default catalog pick | Context | Example hardware |\n|------|--------------|---------|-----------------|\n| 0 \u002F 8 GB unified RAM | Phi-4 Mini (Q4_K_M) | 128K | M1\u002FM2 base (8GB) |\n| 1 \u002F 16 GB unified RAM | Qwen3.5 9B (Q4_K_M) | 32K | M4 Mac Mini (16GB) |\n| 2 \u002F 32 GB unified RAM | Phi-4 14B (Q4_K_M) | 16K | M4 Pro Mac Mini, M3 Max MacBook Pro |\n| 3 \u002F 48 GB unified RAM | Qwen3.5 27B (Q4_K_M) | 32K | M4 Pro (48GB), M2 Max (48GB) |\n| 4 \u002F 64+ GB unified RAM | Qwen3.6 35B-A3B (UD-Q4_K_M) | 128K | M2 Ultra Mac Studio, M4 Max (64GB+) |\n\n### Intel Arc (Linux, SYCL)\n\n| Tier \u002F envelope | Current default catalog pick | Context | Example hardware |\n|------|--------------|---------|------------------|\n| ARC_LITE \u002F 6 GB discrete VRAM | Phi-4 Mini (Q4_K_M) | 128K | Arc A380 |\n| ARC_LITE \u002F 8 GB discrete VRAM | Qwen3.5 9B (Q4_K_M) | 32K | Arc A750 |\n| ARC \u002F 16 GB discrete VRAM | Phi-4 14B (Q4_K_M) | 16K | Arc A770 16GB, newer Arc GPUs |\n\nGemma 4 profile tiers remain in the installer tier maps: E2B on entry hardware, E4B on midrange hardware, 26B-A4B on pro hardware, and 31B on large\u002Fultra hardware.\n\n---\n\n## Bootstrap Mode\n\nNo waiting for large downloads. ODS uses bootstrap mode by default:\n\n1. Downloads a tiny 1.5B model in under a minute\n2. You start chatting immediately\n3. The full model downloads in the background\n4. Hot-swap to the full model when it's ready — zero downtime\n\nThe bootstrap model starts with a 64K context window so Hermes can work during the first session. After the background download finishes, ODS swaps to the full model and restores the Hermes\u002Ffull-model context target.\n\nSkip bootstrap: `.\u002Finstall.sh --no-bootstrap`\n\n---\n\n## Switching Models\n\nThe installer picks a model for your hardware, but you can switch anytime:\n\n```bash\nods model current              # What's running now?\nods model list                 # Show all available tiers\nods model swap T3              # Switch to a different tier\n```\n\nIf the new model isn't downloaded yet, pre-fetch it first:\n\n```bash\n.\u002Fscripts\u002Fpre-download.sh --tier 3    # Download before switching\nods model swap T3                    # Then swap (restarts llama-server)\n```\n\nAlready have a GGUF you want to use? Drop the single `.gguf` file in\n`data\u002Fmodels\u002F`, then open Dashboard -> Models and load the local entry. For\nolder installs or headless maintenance, update `GGUF_FILE` and `LLM_MODEL` in\n`.env`, then restart with the CLI:\n\n```bash\nods restart llm\n```\n\nOr restart the container directly from the installed `ods` directory:\n\n```bash\ndocker compose restart llama-server\n```\n\nRollback is automatic — if a new model fails to load, ODS reverts to your previous model.\n\n---\n\n## Extensibility\n\nODS is designed to be modded. Every service is an extension — a folder with a `manifest.yaml` and a `compose.yaml`. The dashboard, CLI, health checks, and compose stack all discover extensions automatically.\n\n```\nextensions\u002Fservices\u002F\n  my-service\u002F\n    manifest.yaml      # Metadata: name, port, health endpoint, GPU backends\n    compose.yaml       # Docker Compose fragment (auto-merged into the stack)\n```\n\n```bash\nods enable my-service     # Enable it\nods disable my-service    # Disable it\nods list                  # See everything\n```\n\nThe installer itself is modular — 19 library modules, a shared service registry, and 13 ordered phases. Want to add a hardware tier, swap a default model, or skip a phase? Start with the installer architecture map so you update the Linux, macOS, Windows, upgrade, and host-agent writers together.\n\n[Full extension guide](ods\u002Fdocs\u002FEXTENSIONS.md) | [Installer architecture](ods\u002Fdocs\u002FINSTALLER-ARCHITECTURE.md)\n\n---\n\n## ods-cli\n\nThe `ods` CLI manages your entire stack:\n\n```bash\nods status                # Health checks + GPU status\nods list                  # All services and their state\nods logs llm              # Tail logs (aliases: llm, stt, tts)\nods restart [service]     # Restart one or all services\nods start \u002F stop          # Start or stop the stack\n\nods mode cloud            # Switch to cloud APIs via LiteLLM\nods mode local            # Switch back to local inference\nods mode hybrid           # Local primary, cloud fallback\n\nods model swap T3         # Switch to a different hardware tier\nods enable n8n            # Enable an extension\nods disable whisper       # Disable one\n\nods config show           # View .env (secrets masked)\nods preset save gaming    # Snapshot current config\nods preset load gaming    # Restore it\n```\n\n---\n\n## How It Compares\n\nOther tools get you part of the way. ODS gets you the whole way.\n\n| | ODS | Ollama + Open WebUI | LocalAI |\n|---|:---:|:---:|:---:|\n| **Scope** | Full AI stack — inference to agents to workflows | LLM + chat | LLM only |\n| One-command install | Everything, auto-configured | LLM + chat only | LLM only |\n| Hardware auto-detect + model selection | NVIDIA + AMD Strix Halo + Apple Silicon + Intel Arc + CPU\u002Fcloud fallback | No | No |\n| AMD APU unified memory support | Platform-specific accelerated backend, selected by installer | Partial (Vulkan) | No |\n| Autonomous AI agents | Hermes Agent default; OpenClaw legacy opt-in | No | No |\n| Workflow automation | n8n (400+ integrations) | No | No |\n| Voice (STT + TTS) | Whisper + Kokoro | No | No |\n| Image generation | ComfyUI | No | No |\n| RAG pipeline | Qdrant + embeddings | No | No |\n| Extension system | Manifest-based, hot-pluggable | No | No |\n| Multi-GPU | Yes (NVIDIA) | Partial | Partial |\n\n---\n\n## Documentation\n\n| | |\n|---|---|\n| [Quickstart](ods\u002FQUICKSTART.md) | Step-by-step install guide with troubleshooting |\n| [Docs Index](ods\u002Fdocs\u002FREADME.md) | Maintained map for operators, contributors, and reviewers |\n| [Build On ODS](ods\u002Fdocs\u002FBUILD-ON-ODS-SERVER.md) | Forking, custom editions, extension templates, and downstream validation |\n| [Forkability](ods\u002Fdocs\u002FFORKABILITY.md) | How to fork, audit, customize, and independently operate ODS |\n| [Maintainer Runbook](ods\u002Fdocs\u002FMAINTAINER_RUNBOOK.md) | Release, rollback, validation, and operator continuity guidance for maintainers and forks |\n| [High-Risk Change Map](ods\u002Fdocs\u002FHIGH_RISK_CHANGE_MAP.md) | Which changes require focused checks, fleet validation, or release-grade gates |\n| [Headless Setup](ods\u002Fdocs\u002FHEADLESS-SETUP.md) | QR onboarding, first-boot setup, AP mode, mDNS, and local agent access |\n| [Support Matrix](ods\u002Fdocs\u002FSUPPORT-MATRIX.md) | Current platform and GPU support status |\n| [Release Validation](ods\u002Fdocs\u002FRELEASE_VALIDATION.md) | User Green gates and the release-grade fleet\u002Fdistro validation policy |\n| [2.6.0 Release Notes](ods\u002Fdocs\u002FRELEASE_NOTES_2.6.0.md) | Current stable release notes, validation receipt, and known validation boundaries |\n| [Validation Matrix](ods\u002Fdocs\u002FVALIDATION-MATRIX.md) | Sanitized CI, distro lab, and real-hardware fleet release-readiness evidence |\n| [Validation Reproducibility](ods\u002Fdocs\u002FVALIDATION_REPRODUCIBILITY.md) | How forks and operators can reproduce the validation story on their own hardware |\n| [Offline And Mirroring](ods\u002Fdocs\u002FOFFLINE_AND_MIRRORING.md) | Pinning, mirroring, and preserving release artifacts for independent operation |\n| [Installer Trust](ods\u002Fdocs\u002FINSTALLER_TRUST.md) | Inspect-first install paths, ref pinning, and current provenance limits |\n| [Model Management](ods\u002Fdocs\u002FMODEL-MANAGEMENT.md) | Curated and Hugging Face GGUF discovery, verified imports, switching, and recovery |\n| [Hardware Guide](ods\u002Fdocs\u002FHARDWARE-GUIDE.md) | What to buy, tier recommendations |\n| [FAQ](ods\u002FFAQ.md) | Common questions and configuration |\n| [Extensions](ods\u002Fdocs\u002FEXTENSIONS.md) | How to add custom services |\n| [Installer Architecture](ods\u002Fdocs\u002FINSTALLER-ARCHITECTURE.md) | Modular installer deep dive |\n| [Installer Phase Contracts](ods\u002Fdocs\u002FINSTALLER_PHASE_CONTRACTS.md) | Phase ownership, idempotency, failure modes, and validation expectations |\n| [Compose Resolver Contracts](ods\u002Fdocs\u002FCOMPOSE_RESOLVER_CONTRACTS.md) | Rules for compose layers, extensions, backends, ports, and mode overlays |\n| [Changelog](ods\u002FCHANGELOG.md) | Version history and release notes |\n| [Contributing](CONTRIBUTING.md) | How to contribute |\n\n---\n\n## Contributors And Recognition\n\nODS is built by a growing group of contributors across installers, GPU support, dashboard, security, extensions, docs, and release validation. The README keeps the product overview focused; the long-form credits, upstream acknowledgements, and contributor history live in [CONTRIBUTORS.md](CONTRIBUTORS.md).\n\nODS has been recognized by the local AI and developer community, including AMD Featured Developer recognition, selection as a May 2026 AMD Lemonade Developer Challenge winner, and a feature at [(Co)nnect: Philly's AI Ecosystem Summit](https:\u002F\u002Fluma.com\u002Fxdwih64h) at Pennovation Works.\n\n---\n\n## License\n\nApache 2.0 — Use it, modify it, ship it. See [LICENSE](LICENSE).\n\n---\n\n\u003Cdiv align=\"center\">\n\n*Built by [Osmantic](https:\u002F\u002Fgithub.com\u002FOsmantic) and the growing resistance that refuses to rent what should be owned.*\n\n\u003C\u002Fdiv>\n","ODS（Osmantic Deployment System）是一个面向个人用户的本地AI服务器部署系统，旨在将普通PC、Mac或Linux设备快速配置为功能完备的私有AI运行环境。它集成模型推理（支持Ollama等后端）、Web聊天界面、语音交互、智能体编排、RAG文档检索、图像生成（如ComfyUI）及统一管理控制台，并内置隐私保护与运维监控能力。所有组件通过声明式安装自动装配，无需手动配置多个独立服务。适用于家庭实验室、个人开发者本地AI实验、企业内网离线AI应用开发及注重数据隐私的轻量级AI生产场景。",2,"trending"]