[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-105267":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":16,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":17,"compositeScore":18,"rankGlobal":10,"rankLanguage":10,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":24,"readmeContent":25,"aiSummary":10,"trendingCount":15,"starSnapshotCount":15,"syncStatus":26,"lastSyncTime":27,"discoverSource":28},105267,"hunch","undrgrnd-labs\u002Fhunch","undrgrnd-labs","a systemone proxy using embeddings that distills a teacher model","https:\u002F\u002Ftryhunch.dev",null,"Python",292,26,92,0,34,102,80.64,"Apache License 2.0",false,"main",true,[],"2026-11-01 00:13:03","# hunch\n\nhunch is a \"System One\"-style server that answers questions (e.g. \"did this tool call fail?\" or \n\"does this output contain a prompt injection?\"), on _CPUs_ in milliseconds. It learns and then graduates from a slower, more expensive \"teacher\" model and\nkeeps itself up-to-date by checking in, even after it graduates.\n\nIf LLMs are about _thinking hard_, and decision models are about fast _intuition_, hunch is like the fast _reflex_\nsystem you can use for repetitive tasks.\n\nhunch is designed to fit into systems built around tiers of reasoning for answering questions. Frontier LLMs will get\nthe answer right more often, but are cost and speed prohibitive to run on thousands of records or near-real time. \"System One\"-style models\nlike Jev, Clef, or djev are faster, but often larger models with much large memory requirements than hunch and 100s of\nmilliseconds of latency vs 10s.\n\n![tool calls graded by hunch, clef, luna and the costs (hunch is faster and cheaper, luna the slowest and most expensive)](docs\u002Fdemo\u002Fhunch-demo.gif)\n\n*synthetic tool calls, hunch vs clef vs luna (serialized)*\n\n### But not just that!\n\nhunch is designed for fast decisions on high-volume data. Agent traces are one source, but it can be anything: log files,\nuser comments, transcripts... anything you like! (multi-modal support in the hunch proxy is coming soon)\n\nhunch also ships with `hunch.live` - a Python and TypeScript library allowing you to build dynamic `choice`-style questions\nin real-time on both text and images, learning from both a teacher model and user feedback.\n\n\u003Ctable border=0>\n\n\u003Ctr>\u003Ctd>\n  \n![the objects demo naming an orange held up to a webcam, with Clef teaching](docs\u002Fdemo\u002Fobjects.png) \n\n\u003C\u002Ftd>\u003Ctd>\n\n![the gestures demo reading a thumbs up and a smile from a webcam](docs\u002Fdemo\u002Fgestures.png)\n\n\u003C\u002Ftd>\u003C\u002Ftr>\n\n\u003C\u002Ftable>\n\n*webcam demos built on `hunch.live`: [objects](examples\u002Fobjects\u002F) (left) and [gestures](examples\u002Fgestures\u002F) (right)*\n\n### Caveats\n\nhunch works for a lot of questions, but not all questions. hunch is fast because it doesn't think, so that means if your\nquestions need a lot of indirect reasoning, like \"how bad is this failure, and who can fix it?\", the other judges will be a better fit (we have some [question design guidance for this](docs\u002Fquestion-design.md), with a pre-flight check that tells you which of your questions hunch can take over). \nBut don't worry - if hunch never graduates from learning, it keeps passing the questions up to the teacher indefinitely\ninstead of trying to answer something it isn't capable of.\n\n## Ok, I'm hooked!\n\nhunch is built on top of EmbeddingGemma2 - a model that takes multimodal input (text, images, audio) and spits out a\ngiant vector representing semantic concepts from that input.\n\nBefore we tell you more about hunch, some useful things to know:\n\n- **Structured decisions:** a question with a fixed kind of answer: yes or no (`noul`), one of named options (`choice`), or\n  one of ordered levels (`score`). If you're curious, `noul` is Jev's name for a yes\u002Fno question (from Ber`noul`li distribution), meaning \"probability of yes\".\n- **Teacher:** the model you would otherwise call for every question, such as a decision model (Clef, Jev, etc) or an LLM-as-judge (frontier model or otherwise).\n- **Student:** hunch's small classifier for one question, which reads an embedding of the state from EmbeddingGemma 2.\n- **State:** what you're asking the question about.\n- **Graduation:** where hunch starts answering on its own (cross-validation on the\n  teacher's answers shows it agrees with the teacher often enough)\n- **Audit:** after graduation, some of the student's answers also go to the teacher to check up.\n\nhunch trains the student on the teacher's judgements and then takes over once it is confident enough.\n\n## Why\n\n(... and why we built it!)\n\nOne _very_ useful area for decision models is agent trace analysis. Agent loops generate hundreds of lines of prose,\ncode, tool calls. We need to analyze all of these traces, in real-time, for correctness, failure modes, safety or other\nmetrics.\n\nThese feedback systems ask the same small questions over and over, on every single streaming record. The usual answer is an LLM\nor decision model judge: hundreds of milliseconds in the best case, and a model call (which adds up when you bring in smarter models) per question.\n\n_Most_ of those questions are easy _most_ of the time. hunch sits in front of your judge (its teacher) and learns from its answers. hunch\nonly starts answering only once cross-validation shows it agrees with the teacher often enough. It's continually audited\nagainst the teacher afterwards to make sure it's still answering well. If it sees something that's too far outside\nof what it learned, it sends it to the teacher to help it tune itself.\n\nYou keep your judge in the loop, but you call it less!\n\n- **Cheaper and faster.** hunch can take a lot of the load off the more expensive judge. Depending on the\n  difficulty of the question, it might even be nearly 100%.\n- **Meets or beats the teacher alone.** hunch tries to distill the teacher's answer to the question, but the combination\n  of hunch + teacher often beats a teacher or EG2 alone (this depends on the question - see caveats).\n- **No GPU.** EmbeddingGemma 2 is run using a highly-tuned Rust engine for commodity CPUs.\n\n## What's here\n\n- **[hunch](hunch\u002F):** the proxy (including [hunch.live](hunch\u002Fsrc\u002Fhunch\u002Flive.py)).\n- **[eg2cpu](eg2cpu\u002F):** the engine hunch runs on: Google's EmbeddingGemma 2 embedding model, with an optimized CPU\n  engine. Can be used on its own if hunch doesn't meet your needs.\n- **[examples](examples\u002F):** experiments on public datasets (coding-agent traces, InjecAgent, SST-5, PII) that\n  you can test offline.\n- **[docs](docs\u002F):** [designing questions hunch can take over](docs\u002Fquestion-design.md),\n  [performance](docs\u002Fperformance.md) and a [glossary](docs\u002Fglossary.md).\n\n## Performance\n\n- **hunch:** text decisions in 22 ms (p50) on an Apple M5 Pro, for a tool result of about 190\n  tokens. Under load, concurrent requests: 86 requests\u002Fs at 32 concurrent callers, p99\n  466 ms. See [hunch\u002FREADME.md](hunch\u002FREADME.md#performance).\n- **eg2cpu:** encoding a 32-token text to layer 8 (an optimized path hunch often takes), takes about 0.6 ms per text in batches on a\n  24-core Sapphire Rapids server with AMX. Per-hardware numbers are in [docs\u002Fperformance.md](docs\u002Fperformance.md).\n\n## Setup\n\nYou need Rust (1.88 or later) and [uv](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002F). `uv` and `maturin` are used to build eg2cpu's Python extension.\n\n```bash\nuv sync --all-packages\n```\n\nDownload the model:\n\n```bash\ncd examples && uv run download.py embeddinggemma-2 --yes\n```\n\n## Tests\n\n```bash\n(cd eg2cpu && cargo test --release)\nuv run --package eg2cpu pytest -q eg2cpu\u002Ftests\nuv run --package hunch pytest -q hunch\u002Ftests\n```\n\n## Licence\n\nhunch, eg2cpu and the examples are licensed under either of [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE), at\nyour option.\n\nThe examples ship hashes and labels for datasets. Datasets are under various licenses, listed in [examples\u002FREADME.md](examples\u002FREADME.md).\n\nThe EmbeddingGemma 2 checkpoint is Apache-2.0 (not included).\n",2,"2026-10-31 04:26:16","CREATED_QUERY"]