[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-96182":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":15,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":17,"rankGlobal":10,"rankLanguage":10,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":10,"pushedAt":10,"updatedAt":23,"readmeContent":24,"aiSummary":25,"trendingCount":15,"starSnapshotCount":15,"syncStatus":26,"lastSyncTime":27,"discoverSource":28},96182,"Qwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install","MiaAI-Lab\u002FQwen3.8-27B-16gb-NVIDIA-GPUs-one-click-install","MiaAI-Lab","Qwen3.8-27B on 16-32 GB Nvidia GPUs one-click install for Windows \u002F Linux","https:\u002F\u002Fx.com\u002FMiaAI_lab",null,"Python",219,14,1,0,107,53.53,"MIT License",false,"main",true,[],"2026-09-20 04:01:32","\u003Ch1 align=\"center\">Qwen3.8-27B on 16-32 GB Nvidia GPUs one-click install for Windows \u002F Linux\u003C\u002Fh1>\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fintro.png\" alt=\"Qwen3.8-27B one-click install\" width=\"900\" \u002F>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Csub>by \u003Ca href=\"https:\u002F\u002Fx.com\u002FMiaAI_lab\">Mia'a AI Lab\u003C\u002Fa>\u003C\u002Fsub>\n  \u003Cbr>\u003Cbr>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fsponsors\u002FMiaAI-Lab\" target=\"_blank\" rel=\"noopener noreferrer\" style=\"display:inline-block;margin:0 8px;vertical-align:middle;\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FSponsor%20me%20on%20GitHub-181717?style=for-the-badge&logo=githubsponsors&logoColor=white\" alt=\"Sponsor me on GitHub\" height=\"28\" style=\"height:28px;width:auto;vertical-align:middle;border:0;\" \u002F>\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fx.com\u002FMiaAI_lab\" target=\"_blank\" rel=\"noopener noreferrer\" style=\"display:inline-block;margin:0 8px;vertical-align:middle;\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FFollow%20me%20on%20X-000000?style=for-the-badge&logo=x&logoColor=white\" alt=\"Follow Mia on X\" height=\"28\" style=\"height:28px;width:auto;vertical-align:middle;border:0;\" \u002F>\u003C\u002Fa>\n\u003C\u002Fp>\n\nA serving kit for [Qwen\u002FQwen3.8-27B](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3.8-27B) in\n**[turboderp](https:\u002F\u002Fhuggingface.co\u002Fturboderp)**'s EXL3 quants, on one consumer\nNVIDIA card. It picks a quant that fits the card it finds, installs its own Python\nenvironment, downloads the weights, serves an **OpenAI-compatible** endpoint, and\nopens a chat UI. Windows and Linux, same behaviour.\n\nIt started as a 16 GB recipe — the 2.0 bpw quant is still that floor, and still the\none thing here that is not turboderp's own upload ([Mia-AiLab\u002FQwen3.8-27B-EXL3-2.0bpw](https:\u002F\u002Fhuggingface.co\u002FMia-AiLab\u002FQwen3.8-27B-EXL3-2.0bpw),\n`SC_2.00bpw_H3_V3`). Everything from 2.5 bpw up is pulled from\n[turboderp\u002FQwen3.8-27B-exl3](https:\u002F\u002Fhuggingface.co\u002Fturboderp\u002FQwen3.8-27B-exl3)\nby revision. Which one you get is [decided by your VRAM](#what-the-launcher-picks-for-your-gpu),\nat setup, and you can change it any time.\n\n> **Jump to your guide:** [Windows](#windows) · [Linux](#linux)\n\n---\n\n## Before you start (both systems)\n\n| | |\n| --- | --- |\n| GPU | NVIDIA, 12 GB VRAM or more, compute capability 7.5+ (Turing and newer). 16 GB is the size this kit was built around. |\n| Driver | 570 or newer (the default PyTorch build is cu128). |\n| Python | **3.11 or newer, 64-bit.** The only thing you install by hand. |\n| Node | **22.19+** from [nodejs.org](https:\u002F\u002Fnodejs.org\u002F) (current dsh). Older LTS (20) warns `EBADENGINE` and the chat UI may fail. Without Node, `\u002Fv1` still serves; the launcher says what is missing. |\n| Disk | 9.7–22.9 GB per quant (see the table below), plus several GB for the Python environment and PyTorch. |\n\n**Not needed:** CUDA Toolkit, Visual Studio Build Tools, Git. The engine arrives as a\n[prebuilt wheel](#prebuilt-wheels-no-compiler-needed); compiling is the fallback for\nplatforms no wheel covers.\n\nEverything the kit installs stays inside its own folder — `.venv\u002F`, `models\u002F`,\n`logs\u002F`, `apps\u002F`, `.dsh\u002F`. Nothing goes into the system Python and nothing needs\nadministrator rights.\n\n---\n\n## Windows\n\nEverything is in the `windows\\` folder. Run the `.bat` files from Explorer\n(double-click) or from a `cmd` window opened in the kit folder.\n\n### 1. Install — `windows\\START-HERE.bat`\n\nDouble-click it once. It opens a page in your browser and does the whole install\nthere: it shows what it found on the card, offers the model sizes that fit it, then\ninstalls and downloads with a progress bar and a live log. Nothing is asked in the\nconsole.\n\nWhen the download finishes it **loads that model** and hands the page over to the\nchat, so one double-click takes you from nothing to a working chat window.\n\n```\nwindows\\START-HERE.bat              install, then start what was installed\nwindows\\START-HERE.bat --no-start   install only — for fetching a second size\n```\n\nNotes:\n\n* The download is **resumable**. Closing the window, losing the connection or\n  rebooting costs you nothing — it picks up from the byte it stopped at. A model\n  left half-downloaded is shown as such in the menus, and running\n  `windows\\START-HERE.bat` again finishes it. Nothing offers to *start* a model\n  until every weight file is on disk.\n* Prefer the old console questions to the web page? Set `SETUP=console` in `.env`.\n\n### 2. Every day after that — `windows\\start.bat`\n\n```\nwindows\\start.bat          start a model that is already here\nwindows\\start.bat setup    go to setup instead (same as START-HERE.bat)\n```\n\nIt never downloads anything. What it does:\n\n1. **Asks which model**, if more than one size is on disk. Enter takes the one that\n   ran last, and it starts on its own after 45 seconds so an unattended machine\n   still comes up.\n2. **Checks free VRAM** right before the load — it wants the `GPU_MEM_GB` budget\n   from `.env` plus a little margin. If that much is not free it lists the programs\n   holding VRAM (browsers, games, Discord, other AI tools) and waits: Enter\n   re-checks, `c` continues anyway, `q` quits, and it continues on its own after\n   120 seconds. **Take this seriously on Windows** — with too little free VRAM the\n   driver pages the model into system RAM instead of failing, and it then runs many\n   times slower.\n3. **Loads the model** and opens the [chat UI](#the-deepseek-harness) at\n   `http:\u002F\u002F127.0.0.1:3080\u002F`.\n\nIf nothing is installed yet, or nothing finished downloading, it says so and offers\nto run setup for you — double-clicking the wrong one is never a dead end.\n\nTwo files rather than one because they answer two different questions:\n`windows\\start.bat` never downloads, and `windows\\START-HERE.bat --no-start` never\nloads.\n\n### 3. While it runs\n\n* The console window it opened **is** the server. Closing it stops the model.\n* Simplex puts an icon in the notification area — right-click for **Open Simplex**,\n  **Restart the model**, **Show the Simplex folder**, **View the log** and\n  **Quit Simplex**.\n  (`TRAY=no` in `.env` turns it off.)\n* Every launch writes a full transcript to `logs\\`, so a crash that scrolls past is\n  still readable afterwards.\n* The first successful launch adds Start-menu and desktop shortcuts\n  (`SHORTCUTS=no` in `.env` to skip that).\n\n### 4. Stopping — `windows\\stop.bat`\n\n```\nwindows\\stop.bat                 stop both the model and the chat UI\nwindows\\stop.bat --harness-only  leave the model loaded, close the UI\nwindows\\stop.bat --server-only   leave the UI running, unload the model\n```\n\n### 5. Windows troubleshooting\n\n| symptom | what to do |\n| --- | --- |\n| \"Simplex needs Python and cannot find it\" | Install 64-bit Python 3.11+ from [python.org](https:\u002F\u002Fwww.python.org\u002Fdownloads\u002F) and tick **Add python.exe to PATH**, then run the file again. |\n| Anything else | `windows\\simplex.bat doctor` — see [below](#simplex-doctor). |\n| The model loads but crawls | Free VRAM (the check above told you what is holding it), or lower `CONTEXT_SIZE` \u002F `GPU_MEM_GB` in `.env`. |\n| \"Images: off\" in the Ready box | The vision tower did not fit next to your context. Lower `CONTEXT_SIZE` and restart, or pick a smaller quant. |\n| The window closed and you missed the error | It is in `logs\\` — newest file. `windows\\simplex.bat logs` prints the tail. |\n| You want to start completely over | `reset_new_user.bat` in the kit root deletes the weights, the venv, `.env`, the logs and the shortcuts, and keeps every tracked file. It asks you to type `RESET` first. |\n\n---\n\n## Linux\n\nEverything is in the `linux\u002F` folder. Run the scripts from the **kit root**; they\nfind their own way regardless of where you call them from.\n\nIf the files arrived without their execute bit (a zip, a copy off Windows), run them\nas `bash linux\u002Fsetup.sh` instead of `.\u002Flinux\u002Fsetup.sh`, or `chmod +x linux\u002F*.sh linux\u002Fsimplex` once.\n\n### 1. Install — `.\u002Flinux\u002Fsetup.sh`\n\n```bash\n.\u002Flinux\u002Fsetup.sh\n```\n\nIt creates `.env` from `.env.example` on the first run, asks the profile questions\n**in the terminal** (`tools\u002Fprofiles.py`) — there is no setup page on Linux — builds\n`.venv`, installs PyTorch and the engine, downloads the weights, and stops. It does\nnot load a model.\n\nThe download is resumable: interrupt it and run `.\u002Flinux\u002Fsetup.sh` again to carry on.\n\n### 2. Every day after that — `.\u002Flinux\u002Fstart.sh`\n\n```bash\n.\u002Flinux\u002Fstart.sh                 pick a downloaded model and serve it\n.\u002Flinux\u002Fstart.sh --no-harness    serve \u002Fv1 only, no chat UI\n.\u002Flinux\u002Fstart.sh -b              run in the background, output in logs\u002F\n.\u002Flinux\u002Fstart.sh --status        is a backgrounded one running?\n```\n\nIt lists the models that finished downloading and asks which one (Enter is the one\nused last; it auto-picks after 45 seconds), then serves:\n\n```\nhttp:\u002F\u002Flocalhost:8888\u002Fv1     the OpenAI-compatible API\nhttp:\u002F\u002F127.0.0.1:3080\u002F       the chat UI\n```\n\n`-b` is the honest equivalent of the Windows tray: it detaches, writes to\n`logs\u002Fsimplex-*.log`, and tells you where that log is and how to stop it. First-run\nsetup and the model menu still happen — written to the log instead of the screen.\n\n**On a box with no desktop session** `webbrowser` has nothing to open, so the chat\naddress is printed for you to copy. Take the whole thing, **token and all** — see\n[the note on the token](#the-first-address-is-not-the-plain-one).\n\nThere is **no free-VRAM preflight on Linux** (that check is Windows-specific,\nbecause Windows silently spills to system RAM instead of failing). If a load fails\nwith `Insufficient VRAM in split for model and cache`, lower `CONTEXT_SIZE` or\n`GPU_MEM_GB` in `.env`, or close what is holding the card.\n\nNo tray icon and no desktop shortcuts either — those are Windows.\n\n### 3. Stopping — `.\u002Flinux\u002Fstop.sh`\n\n```bash\n.\u002Flinux\u002Fstop.sh                 stop both\n.\u002Flinux\u002Fstop.sh --harness-only  leave the model loaded\n.\u002Flinux\u002Fstop.sh --server-only   leave the chat UI running\n```\n\n### 4. Linux troubleshooting\n\n| symptom | what to do |\n| --- | --- |\n| `bash: .\u002Flinux\u002Fstart.sh: Permission denied` | `chmod +x linux\u002F*.sh linux\u002Fsimplex`, or call it as `bash linux\u002Fstart.sh`. |\n| `$'\\r': command not found` | The checkout has CRLF endings. `.gitattributes` prevents this; re-clone, or `sed -i 's\u002F\\r$\u002F\u002F' linux\u002F*.sh linux\u002Fsimplex`. |\n| Anything else | `.\u002Flinux\u002Fsimplex doctor` — see [below](#simplex-doctor). |\n| `Insufficient VRAM in split for model and cache` | Lower `CONTEXT_SIZE` or `GPU_MEM_GB` in `.env`, or run `.\u002Flinux\u002Fsimplex setup` and pick a smaller quant. |\n| It compiled the engine for 20 minutes | No prebuilt wheel matched your CUDA line, torch version or Python. See [Prebuilt wheels](#prebuilt-wheels-no-compiler-needed). |\n| aarch64 \u002F GB10 | No prebuilt engine wheel exists on any CUDA line, so it compiles. The script keeps cu130 there and sets `TORCH_CUDA_ARCH_LIST=12.0;12.1` for you. |\n\n---\n\n## One command, both systems\n\nThe files above are the double-click doors. Every verb, on either system, is\n`simplex` — the same program (`tools\u002Fcli.py`), so the two cannot drift apart:\n\n| Linux | Windows |\n| --- | --- |\n| `.\u002Flinux\u002Fsimplex \u003Cverb>` | `windows\\simplex.bat \u003Cverb>` |\n\n```\nsimplex setup                   install the environment and fetch a model\nsimplex start                   load a model and serve it\nsimplex start --no-harness      ...serving \u002Fv1 only\nsimplex start -b                ...in the background, log in logs\u002F\nsimplex start -p 9000           ...on another port, just this once\nsimplex stop                    stop both\nsimplex stop --harness-only     ...and detach the UI, model still loaded\nsimplex restart                 stop, then start\nsimplex status                  what is running, which model, which ports\nsimplex status --json           the same, for scripts\nsimplex logs -f                 follow the launcher log\nsimplex models                  what is on disk, and what is half-downloaded\nsimplex harness start           attach the UI to a server already running\nsimplex harness stop|status|open|settings\nsimplex doctor                  check this machine before blaming the model\n```\n\n`simplex` with no verb prints the help and then the status. Every verb takes\n`--help`. It is not on your `PATH` — run it from the kit folder.\n\n### simplex doctor\n\nThe first thing to run when something is wrong. It checks the Python version, the\nvenv and the engine version inside it, the driver and the card, Node, both ports and\nwho holds them, the `.env` values that have to be valid, whether the weights are all\nthere, and the free disk.\n\n### With or without the chat UI\n\n`UI=` in `.env` is the standing answer (`browser`, `server` or `no`);\n`--harness` \u002F `--no-harness` overrides it for one run, on `simplex start`,\n`linux\u002Fstart.sh` and `windows\\start.bat` alike. The UI is also a verb of its own, so\nit can be attached to a model that is already loaded, or taken away without\nunloading one.\n\n---\n\n## What the launcher picks for your GPU\n\nSetup (`windows\\START-HERE.bat` \u002F `.\u002Flinux\u002Fsetup.sh`, or `simplex setup`, or\n`PROFILE=ask` in `.env`) runs `tools\u002Fprofiles.py`. It reads the card's VRAM with\n`nvidia-smi`, computes what fits under a budget of *VRAM − max(1.3 GB, 8 %)*, and\noffers the sizes that fit. Enter takes the recommendation. If a model is already\ndownloaded, \"keep current\" is the default, so an unattended start never triggers a\nsurprise download.\n\nThe choice is written into `.env` (`MODEL_DIR`, `HF_TARGET_REPO`, `HF_REVISION`,\n`MODEL_ID`, `CONTEXT_SIZE`, `CACHE_QUANT`, `GPU_MEM_GB`, `VISION`) and everything\ndownstream follows it.\n\n| VRAM | what it offers (**bold** = pre-selected) |\n| --- | --- |\n| 12 GB | **2.0 bpw @ 33k**, text-only — the floor, and the whole menu |\n| 16 GB | 3.5 bpw @ 78k text-only · 3.0 bpw @ 118k with images · **2.5 bpw @ 176k with images** · 2.0 bpw @ 229k with images |\n| 24 GB | 6.0 bpw @ 84k text-only · 5.0 bpw @ 180k with images · **4.0 bpw @ 262k with images** · 3.5 and below at 262k with images |\n| 32 GB+ | the same menu as 24 GB — the top two rows are capped at what a prefill has actually survived, not at what the card could hold |\n\nKV cache is **int4** on every profile the planner writes — measured within 0.001 KL\nof fp16, with no hardware requirement, so it runs on every supported GPU. (The\n`.env.example` template still ships `CACHE_QUANT=8,4`, which is the old hand-tuned\n2.0 bpw baseline below; picking a profile overwrites it.)\n\nEach row's download size and the exact context it plans are printed by the planner\nitself, and it will do that for any card without you owning one:\n\n```\nWindows:  .venv\\Scripts\\python.exe tools\\profiles.py --list --vram 16\nLinux:    .venv\u002Fbin\u002Fpython tools\u002Fprofiles.py --list --vram 16\n```\n\n### Where those numbers come from\n\nWhere a real prefill has been run at a stated budget, the menu offers what was\nmeasured rather than what the formula computes. The context was grown on a 14.7 GB\nbudget until a prefill failed (2026-09-06), and the formula had been leaving a lot\non the table:\n\n| budget | quant | planner offered | measured, text | measured, images |\n| --- | --- | --- | --- | --- |\n| 14.7 GB | 3.5 | 0 | 77824 | 41984 |\n| 14.7 GB | 3.0 | 69888 | 148480 | 117760 |\n| 14.7 GB | 2.5 | 186368 | 212224 | 176128 |\n| 22.1 GB | 6.0 | 8960 | 83712 | 57088 |\n| 22.1 GB | 5.0 | 163072 | 204800 | 179712 |\n| 22.1 GB | 4.0 | 262144 | 262144 | 262144 |\n\nThe formula's flat 2.6 GB overhead is a bound over every quant, so on any one of\nthem it is slack — 3.5 bpw is the extreme case, priced out of a 16 GB card entirely\nby a formula that the card then ran at 78k tokens, and 6.0 bpw is not far behind at\n9k against 84k. Above the budget a row was measured under, the formula takes over\nagain; below it, the measurement only ever lowers the answer.\n\nTwo rows are also *capped* at their measurement: nothing has ever prefilled past\n204800 tokens on 5.0 bpw or 83712 on 6.0 bpw, at any budget, so neither plans past\nit. (Their older ceilings — 183296 for 5.0, \"nothing survived\" for 6.0 — came from a\nrun with ~40 other processes on the card, and a clean run at a *tighter* budget beat\nboth, which is how you tell contention from a ceiling.)\n\n### How the default is chosen\n\nBold above is the best quality that still has real context (≥ 128k), not the longest\ncontext. On a 16 GB card that is the 2.5 bpw row — 3.0 bpw is the better model, but\nit fits 118k there and only text-only, against a measured 176k with images one rung\ndown. The pick also keeps images where it can: on a 24 GB card 5.0 bpw clears 128k\nonly by dropping the vision tower, so the default steps one rung down to 4.0 bpw,\nwhich holds native context with images. One rung, never more — and answering\n\"no images\" puts 5.0 bpw back.\n\nQuants other than the 2.0 bpw baseline are pulled from turboderp's branches\n(`HF_REVISION`); their vision towers are unquantised (0.87 GB measured), which is\nwhy images are off on the tight profiles. The quality words come from turboderp's\nmean-KL-vs-bf16 figures: 2.0 → 0.35 *fair*, 2.5 → 0.30 *good*, 3.0 → 0.11 *better*,\n3.5 → 0.08 *very good*, 4.0 → 0.05 *very good*, 5.0 → 0.014 *excellent*,\n6.0 → 0.007 *near-lossless*.\n\n### The original 16 GB baseline\n\nA 16 GB board typically has about **14.7 GB free** after the driver. These are the\nhand-tuned settings the 2.0 bpw quant was validated on, and what `.env.example`\nstill ships:\n\n| Knob | Value | Why |\n| --- | --- | --- |\n| `GPU_MEM_GB` | `14.7` | Process cap matching ~14.7 GB free on a 16 GB card |\n| `CONTEXT_SIZE` | `199936` | ~200k tokens (must be a multiple of the 256-token page size) |\n| `CACHE_QUANT` | `8,4` | int8 K \u002F int4 V |\n| `DRAFT` | `mtp` (default) | MTP head inside the checkpoint; ~50 MB extra weights |\n\nMeasured at load under that cap: CUDA **allocated 12.35 GiB**, **reserved 13.37\nGiB**; native `CONTEXT_SIZE=262144` **fails to boot**\n(`Insufficient VRAM in split for model and cache`). Weights on disk are ~9.7 GB; the\nrest is KV (16 full-attention layers), the MTP draft cache, GDN recurrent state and\nCUDA workspace.\n\n**If you have more than 16 GB:** let the profile planner do it — it already knows.\nBy hand, a 24 GB card takes `CONTEXT_SIZE=262144` and `GPU_MEM_GB=22`. Do not do\nthat on 16 GB.\n\n---\n\n## Chat with the model\n\n### The DeepSeek Harness\n\nThe kit serves the model. What you talk to is\n**[DeepSeek Harness](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002Fdeepseek-harness)** (`dsh`, MIT),\nstarted as a second process once the model is loaded and answering, at\n**`http:\u002F\u002F127.0.0.1:3080\u002F`**. `windows\\start.bat` and `.\u002Flinux\u002Fstart.sh` open it for\nyou; `http:\u002F\u002F127.0.0.1:8888\u002F` is a small page saying where everything is, which\nforwards there as soon as the harness answers.\n\nNone of it is vendored here. It is a Node application, so the launcher runs it with\n`npx` and npm caches it after the first run — which is why **Node** is on the\nrequirements list. No Node, no harness: the launcher says so, and `\u002Fv1` keeps\nserving every other client. `UI=no` in `.env` skips it entirely.\n\n**It configures itself against whatever loaded.** Before starting the harness,\n`tools\u002Fdsh.py` asks the running server on `\u002Fv1\u002Fmodels` what it actually is — the\nmodel id, the context window, whether the vision tower fit, and which reasoning\nlevels this chat template accepts *and acts on* — and writes that as a provider\nroute into `.dsh\u002Fsettings.yaml`, naming it as the default the picker opens on. So\nthe model, its context meter, its image support and its effort menu are right on the\nfirst launch with nothing typed into a form. After a fallback (no room for the\nvision tower, say) the harness is told what happened rather than what `.env` hoped\nfor. Switching quants rewrites the same file; the harness re-reads it per request,\nso it never needs restarting.\n\nThose keys are yours the moment you edit them. The launcher keeps a copy of what it\nlast wrote beside the file and stops generating as soon as they differ, so a\nhand-tuned route survives every restart. Delete the file to get a fresh one.\nEverything else in it is read past and written back untouched. Every field it\naccepts is in\n[dsh's configuration catalog](https:\u002F\u002Fgithub.com\u002Fdeepseek-ai\u002Fdeepseek-harness\u002Fblob\u002Fmaster\u002Fdocs\u002Fconfig-catalog.md).\n\n#### The first address is not the plain one\n\ndsh authenticates the browser with a token it mints fresh on every launch and prints\nonce, as `dsh web: http:\u002F\u002F127.0.0.1:3080\u002F?token=...`. Opening **that** sets a cookie\ngood for thirty days and redirects to a clean `\u002F`; arriving at the bare address\nwithout it answers *\"dsh web authentication required; reopen the URL printed by dsh\nweb\"*. So the launcher reads the address off dsh's own output rather than composing\nit from the port, and `http:\u002F\u002F127.0.0.1:8888\u002F` forwards through `\u002Fharness`, which\nknows the current one. If you need it by hand, it is in the launcher window and in\n`logs\u002F`.\n\nThat route only answers a browser on this computer, even with `HOST=0.0.0.0`: the\ntoken is a session on an agent that runs commands here.\n\n| | |\n| --- | --- |\n| Harness | `http:\u002F\u002F127.0.0.1:3080\u002F` (`SIMPLEX_HARNESS_PORT` in `.env`; do not use `DSH_PORT` there) |\n| Version | `DSH_VERSION` in `.env`, pinned; `latest` follows the newest |\n| Its home | `.dsh\u002F` in the kit folder — settings, credentials, profiles, plugins |\n| Run it alone | From the **kit root** (not `C:\\Windows\\System32`): `python tools\u002Fdsh.py --open`. PowerShell: `Set-Location -LiteralPath \u003Ckit>`; cmd: `cd \u002Fd \u003Ckit>`. `cd \u002Fd` is not valid in PowerShell. |\n| Just the settings | `tools\u002Fdsh.py --settings-only` |\n\nThe harness binds loopback only and **refuses to bind `0.0.0.0` at all**: its agent\nruns commands on this PC and there is no login. To reach it from a phone, put a\nproxy in front of it rather than opening the port — `tailscale serve --bg 3080`.\nNote that the shipped `HOST=0.0.0.0` already exposes `\u002Fv1` (the API, not the\nharness) to whatever network you are on; set `HOST=127.0.0.1` if that network is not\nyours.\n\nIts workspace, approval policy, tools, MCP servers and plugins are all its own — see\n[its documentation](https:\u002F\u002Fdeepseek-harness.github.io\u002Fdeepseek-harness\u002F). Pick a\nworkspace folder in it before the first message.\n\n### Any other OpenAI client\n\n`\u002Fv1` is a plain OpenAI endpoint, so nothing about the harness is compulsory. Leave\n`windows\\start.bat` \u002F `.\u002Flinux\u002Fstart.sh` running and point a client at it. There is\n**no API key**; many apps still require a dummy value such as `local`.\n\n| | |\n| --- | --- |\n| Base URL | `http:\u002F\u002F127.0.0.1:8888\u002Fv1` (or host `http:\u002F\u002F127.0.0.1:8888` if the app appends `\u002Fv1` itself) |\n| API key | `local` (ignored) |\n| Model id | whatever `MODEL_ID` in `.env` says — e.g. `qwen3.8-27b-exl3-2.5bpw` for the 2.5 bpw quant. `simplex status` prints it, and so does `GET \u002Fv1\u002Fmodels`. |\n\nChatbox, Open WebUI, Continue, Cursor's custom endpoint, Cherry Studio (below), a\n`curl`, an SDK, another machine on your network — all of them work against that base\nURL, at the same time as the harness does. Open WebUI is stronger if you want a big\ntools\u002FRAG UI and are fine running Docker.\n\n**Tool calling.** Send OpenAI `tools` (function name + JSON schema) on\n`POST \u002Fv1\u002Fchat\u002Fcompletions`. The model emits Qwen XML; the server parses it into\n`tool_calls`. Your app must run the function and POST a follow-up with\n`role: \"tool\"` (and the previous assistant `tool_calls`). `tool_choice` of `auto`,\n`required`, or a named function is supported.\n\n```bash\ncurl http:\u002F\u002F127.0.0.1:8888\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\n    \"model\": \"qwen3.8-27b-exl3-2.5bpw\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Tel Aviv?\"}],\n    \"tools\": [{\n      \"type\": \"function\",\n      \"function\": {\n        \"name\": \"get_weather\",\n        \"description\": \"Current weather for a city\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\"city\": {\"type\": \"string\"}},\n          \"required\": [\"city\"]\n        }\n      }\n    }]\n  }'\n```\n\nPlain chat (no tools):\n\n```bash\ncurl http:\u002F\u002F127.0.0.1:8888\u002Fv1\u002Fchat\u002Fcompletions \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"model\":\"qwen3.8-27b-exl3-2.5bpw\",\"messages\":[{\"role\":\"user\",\"content\":\"Hi\"}]}'\n```\n\n**Images.** The quants keep Qwen3.8's vision tower and the server loads it by\ndefault (`VISION=auto`), so you can send OpenAI `image_url` content parts — `data:`\nURLs or http(s) links. Pictures are downscaled to `IMAGE_MAX_PIXELS`\n(1 MP ≈ 1024 prompt tokens) before encoding. If the tower does not fit next to your\ncontext under the VRAM cap, the Ready box says `Images: off` and the server keeps\nrunning text-only — lower `CONTEXT_SIZE` and restart, or pick a quant whose profile\nsays images are on. Video is not supported.\n\nDefaults: temperature 0.6, top-p 0.95, top-k 20, thinking on. One request at a time;\nextras queue.\n\n### Cherry Studio (optional, off by default)\n\nThe kit used to ship [Cherry Studio](https:\u002F\u002Fgithub.com\u002FCherryHQ\u002Fcherry-studio) as\nits chat app. It is still wired up for anyone who wants Cherry's assistants,\nknowledge bases and MCP servers — set `CHERRY_AUTOSTART=ask` (or `yes`) in `.env`:\n\n* The launcher downloads the pinned **portable** build (v2.0.10, ~285 MB, once) into\n  `apps\u002Fcherry-studio\u002F`. Nothing is installed system-wide; Cherry keeps its data in\n  `apps\u002Fcherry-studio\u002Fdata\u002F`.\n* The first time, Cherry opens and closes once by itself to create that data folder.\n  The kit then writes its configuration straight into Cherry's store: provider\n  **Simplex (local)** → `http:\u002F\u002F127.0.0.1:8888\u002Fv1`, key `local`, the model id from\n  `.env` (tool calling + image input on), set as the default chat model and on the\n  default assistant, onboarding skipped. Usage analytics is switched off\n  (Cherry → Settings → Privacy to change).\n* After the **Ready** box: *\"Open Cherry Studio and start chatting now? [Y\u002Fn]\"*.\n  Enter\u002F`y` opens it (the portable build unpacks for ~10–20 s), `n` or no answer\n  within 90 s leaves it closed. Keep the server window open while chatting.\n* Changing `PORT` in `.env` re-points the provider on the next start. Open Cherry\n  later without the prompt: `.venv\\Scripts\\python.exe tools\\cherry.py open` on\n  Windows, `.venv\u002Fbin\u002Fpython tools\u002Fcherry.py open` on Linux (`status` instead of\n  `open` shows what the kit thinks).\n* `.env` knobs: `CHERRY_AUTOSTART=ask|yes|no` (`no` also skips the download),\n  `CHERRY_VERSION` (pinned; the store layout is checked against v2.0.x),\n  `CHERRY_EXE=\u003Cpath>` to use a Cherry Studio you already installed — the kit then\n  only sends Cherry's official import link (`cherrystudio:\u002F\u002Fproviders\u002Fapi-keys`),\n  you confirm the popup and add the model id under the new provider.\n* **Web search and tools are on by default.** The default assistant gets\n  `web_search` + `web_fetch` as function tools (Cherry's stock keyless search\n  provider, Exa MCP at `mcp.exa.ai`; change it under Settings → Web Search), runs\n  MCP in *auto* mode, and the kit installs Cherry's keyless builtin MCP servers\n  `@cherry\u002Ffetch` and `@cherry\u002Fsequentialthinking`. Tune with\n  `CHERRY_WEB_SEARCH=1|0` and `CHERRY_MCP_SERVERS=` (also `@cherry\u002Fpython`,\n  `@cherry\u002Fbrowser`; empty = none) in `.env` — re-applied on the next start when you\n  change them. New assistants you create in Cherry start with Cherry's own defaults\n  (web search off) unless you copy the default one.\n* Cherry Studio is [AGPL-3.0](https:\u002F\u002Fgithub.com\u002FCherryHQ\u002Fcherry-studio\u002Fblob\u002Fmain\u002FLICENSE)\n  (see its README for the commercial-use terms); the kit downloads the official\n  release binary and does not redistribute it.\n\n---\n\n## Prebuilt wheels: no compiler needed\n\nCompiling the ExLlamaV3 CUDA kernels is the slowest and most fragile part of setup:\nit wants the CUDA Toolkit and Visual Studio Build Tools, several GB of downloads that\nhave nothing to do with chatting to a model. Nobody has to do it, because the engine\npublishes wheels itself.\n\nSimplex looks for one in this order:\n\n1. **`wheels\u002F`** in the kit folder — what you copy off a USB stick.\n2. **The engine's own release** — `turboderp-org\u002Fexllamav3` attaches a wheel per\n   (CUDA line × torch version × Python). This is the normal path and needs no\n   configuration.\n3. **`WHEEL_INDEX`** in `.env` — one or more `pip --find-links` targets (a GitHub\n   Releases page, a file share, an internal index).\n4. **PyPI**, which has `triton-windows` but not `exllamav3`.\n5. **Compiling from source**, for the cases none of the above covers — a CUDA line\n   or platform the engine has no build for (aarch64\u002FGB10), or no route to\n   github.com.\n\nStep 2 resolves to one exact URL rather than pointing pip at the release page, and\nthat distinction matters. The CUDA line and torch version live in the wheel's *local\nversion* (`1.4.4+cu128.torch2.10.0`), which pip does not match against anything:\ngiven `--find-links` it filters on the Python and platform tags only, then takes the\nhighest version string. A torch 2.10 environment would be handed the torch 2.11\nbuild, and the failure arrives later as an undefined-symbol `ImportError` that reads\nlike a corrupt install. So the launcher installs torch first, asks the venv what it\nactually got, and names the one wheel that fits.\n\nThis is also why the default PyTorch index is **cu128**: the engine builds for cu128\nand cu132 only, so torch from any other line means no wheel exists and everyone\ncompiles. cu128 covers Blackwell and needs driver 570+. (`TORCH_INDEX_URL` in `.env`\noverrides it. aarch64 keeps cu130, since no engine wheel exists there on any line.)\n\nA wheel is only used when its Python, ABI and platform tags match the interpreter it\nis going into, so a `cp313` wheel can never land in a `cp312` environment. Check what\nwould be picked:\n\n```\nWindows:  .venv\\Scripts\\python.exe tools\\wheels.py\nLinux:    .venv\u002Fbin\u002Fpython tools\u002Fwheels.py\n```\n\nThat prints the venv's tags, the torch version and CUDA line found, and the wheel it\nwould install. `wheels\u002FREADME.md` covers the override cases and the recipe for\nbuilding one yourself.\n\n**Engine version.** This kit requires **ExLlamaV3 v1.4.4** — it is what the quantized\nvision tower needs, and what the kit is validated against. PyPI skips 1.4.4\n(`1.4.2` → `1.4.5`), so the launchers install the git tag, and both start paths\nrefuse to run against any other version. Engine:\n[ExLlamaV3](https:\u002F\u002Fgithub.com\u002Fturboderp-org\u002Fexllamav3).\n\n---\n\n## Installing on a Windows PC as an app\n\nUnzip (or clone) anywhere, double-click `windows\\START-HERE.bat`, then\n`windows\\start.bat` from then on. There is no system-wide install and no\nadministrator rights are needed; the first successful launch adds the Start-menu and\ndesktop shortcuts for you (`SHORTCUTS=no` in `.env` to skip that).\n\nWhether the browser offers an **Install** button for the chat page depends on the\nfront end you use, not on this kit. Installing only ever works on `localhost` or over\nHTTPS (a secure context); over a plain `http:\u002F\u002F` LAN address browsers refuse to\nregister a service worker, so there the page stays an ordinary one.\n\n---\n\n## Configuration\n\n`.env` in the kit folder, created from `.env.example` on the first run of either\nsystem. It is read as a `key=value` list, not executed, so values with spaces are\nfine. The ones you are most likely to touch:\n\n| key | default | what it does |\n| --- | --- | --- |\n| `MODEL_DIR` | set by setup | which downloaded model to load |\n| `CONTEXT_SIZE` | set by setup | tokens; must be a multiple of 256 |\n| `CACHE_QUANT` | set by setup | `4` (int4), `8,4`, `none`, or `k,v` |\n| `GPU_MEM_GB` | set by setup | the process's VRAM budget |\n| `VISION` | `auto` | `off` to skip the vision tower |\n| `PORT` | `8888` | the OpenAI API port |\n| `HOST` | `0.0.0.0` | set to `127.0.0.1` to keep `\u002Fv1` off your network |\n| `UI` | `browser` | `browser`, `server`, or `no` |\n| `SIMPLEX_HARNESS_PORT` | `3080` | the chat UI's port. Do **not** set `DSH_PORT` in `.env` — current dsh treats that key in a file as fatal and the harness never binds |\n| `DRAFT` | `mtp` | `none` turns off speculative decoding |\n| `SETUP` | `browser` | `console` for terminal questions on Windows |\n| `TRAY` | `auto` | Windows notification-area icon; `no` to skip |\n| `SHORTCUTS` | `auto` | Windows shortcuts; `no` to skip |\n| `HF_TOKEN` | — | only needed for gated repos |\n\n`.venv\u002F`, `models\u002F`, `logs\u002F`, `apps\u002F`, `.dsh\u002F` and `.env` stay on your machine and\nare not part of the git tree.\n\n---\n\n## Simplex, the UI this kit used to ship\n\nUp to this version the kit served its own chat and agent UI in the server process.\nThat UI is now **Simplex**, a standalone project: nothing in it was specific to this\nmodel or this server, and it talks to any OpenAI-compatible endpoint — including this\none, at `http:\u002F\u002F127.0.0.1:8888\u002Fv1`. Its conversations, projects and providers moved\nwith it, so an existing install picks up where it left off. Run its own launcher\nbeside this one and set `UI=no` here if you want it back in place of the harness, or\nrun both.\n\n## License\n\nApache-2.0 (inherited from the base model). Kit scripts: [MIT](LICENSE).\n","这是一个面向消费级NVIDIA显卡（16–32GB VRAM）的Qwen3.8-27B大语言模型轻量化部署工具包，支持Windows\u002FLinux一键安装。核心功能包括自动识别GPU显存并选择适配的EXL3量化版本（2.0–3.5 bpw）、自建隔离Python环境、自动下载模型权重、提供OpenAI兼容API服务端点，并内置本地聊天UI。技术特点为零CUDA编译依赖（采用预编译wheel）、全本地化部署（所有文件限于项目目录内）、基于turboderp优化的EXL3量化格式。适用于个人开发者、研究人员在单卡环境下快速本地运行和测试Qwen3.8-27B模型。",2,"2026-09-11 02:30:13","CREATED_QUERY"]