[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-3663":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},3663,"qmd","tobi\u002Fqmd","tobi","mini cli search engine for your docs, knowledge bases, meeting notes, whatever. Tracking current sota approaches while being all local","",null,"TypeScript",26424,1660,89,53,0,47,319,1759,247,119.66,"MIT License",false,"main",true,[],"2026-06-12 04:00:18","# QMD - Query Markup Documents\n\nAn on-device search engine for everything you need to remember. Index your markdown notes, meeting transcripts, documentation, and knowledge bases. Search with keywords or natural language. Ideal for your agentic flows.\n\nQMD combines BM25 full-text search, vector semantic search, and LLM re-ranking—all running locally via node-llama-cpp with GGUF models.\n\n![QMD Architecture](assets\u002Fqmd-architecture.png)\n\nYou can read more about QMD's progress in the [CHANGELOG](CHANGELOG.md).\n\n## Quick Start\n\n```sh\n# Install globally (Node or Bun)\nnpm install -g @tobilu\u002Fqmd\n# or\nbun install -g @tobilu\u002Fqmd\n\n# Or run directly\nnpx @tobilu\u002Fqmd ...\nbunx @tobilu\u002Fqmd ...\n\n# Create collections for your notes, docs, and meeting transcripts\nqmd collection add ~\u002Fnotes --name notes\nqmd collection add ~\u002FDocuments\u002Fmeetings --name meetings\nqmd collection add ~\u002Fwork\u002Fdocs --name docs\n\n# Add context to help with search results, each piece of context will be returned when matching sub documents are returned. This works as a tree. This is the key feature of QMD as it allows LLMs to make much better contextual choices when selecting documents. Don't sleep on it!\nqmd context add qmd:\u002F\u002Fnotes \"Personal notes and ideas\"\nqmd context add qmd:\u002F\u002Fmeetings \"Meeting transcripts and notes\"\nqmd context add qmd:\u002F\u002Fdocs \"Work documentation\"\n\n# Generate embeddings for semantic search\nqmd embed\n\n# Search across everything\nqmd search \"project timeline\"           # Fast keyword search\nqmd vsearch \"how to deploy\"             # Semantic search\nqmd query \"quarterly planning process\"  # Hybrid + reranking (best quality)\n\n# Get a specific document\nqmd get \"meetings\u002F2024-01-15.md\"\n\n# Get a document by docid (shown in search results)\nqmd get \"#abc123\"\n\n# Get multiple documents by glob pattern\nqmd multi-get \"journals\u002F2025-05*.md\"\n\n# Search within a specific collection\nqmd search \"API\" -c notes\n\n# Export all matches for an agent\nqmd search \"API\" --all --files --min-score 0.3\n```\n\n### Using with AI Agents\n\nQMD's `--json` and `--files` output formats are designed for agentic workflows:\n\n```sh\n# Get structured results for an LLM\nqmd search \"authentication\" --json -n 10\n\n# List all relevant files above a threshold\nqmd query \"error handling\" --all --files --min-score 0.4\n\n# Retrieve full document content\nqmd get \"docs\u002Fapi-reference.md\" --full\n```\n\n### MCP Server\n\nAlthough the tool works perfectly fine when you just tell your agent to use it on the command line, it also exposes an MCP (Model Context Protocol) server for tighter integration.\n\n**Tools exposed:**\n- `query` — Search with typed sub-queries (`lex`\u002F`vec`\u002F`hyde`), combined via RRF + reranking\n- `get` — Retrieve a document by path or docid (with fuzzy matching suggestions)\n- `multi_get` — Batch retrieve by glob pattern, comma-separated list, or docids\n- `status` — Index health and collection info\n\n**Claude Desktop configuration** (`~\u002FLibrary\u002FApplication Support\u002FClaude\u002Fclaude_desktop_config.json`):\n\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": {\n      \"command\": \"qmd\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\n**Claude Code** — Install the plugin (recommended):\n\n```bash\nclaude plugin marketplace add tobi\u002Fqmd\nclaude plugin install qmd@qmd\n```\n\nOr configure MCP manually in `~\u002F.claude\u002Fsettings.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"qmd\": {\n      \"command\": \"qmd\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\n#### HTTP Transport\n\nBy default, QMD's MCP server uses stdio (launched as a subprocess by each client). For a shared, long-lived server that avoids repeated model loading, use the HTTP transport:\n\n```sh\n# Foreground (Ctrl-C to stop)\nqmd mcp --http                    # localhost:8181\nqmd mcp --http --port 8080        # custom port\n\n# Background daemon\nqmd mcp --http --daemon           # start, writes PID to ~\u002F.cache\u002Fqmd\u002Fmcp.pid\nqmd mcp stop                      # stop via PID file\nqmd status                        # shows \"MCP: running (PID ...)\" when active\n```\n\nThe HTTP server exposes two endpoints:\n- `POST \u002Fmcp` — MCP Streamable HTTP (JSON responses, stateless)\n- `GET \u002Fhealth` — liveness check with uptime\n\nLLM models stay loaded in VRAM across requests. Embedding\u002Freranking contexts are disposed after 5 min idle and transparently recreated on the next request (~1s penalty, models remain loaded).\n\nPoint any MCP client at `http:\u002F\u002Flocalhost:8181\u002Fmcp` to connect.\n\n### SDK \u002F Library Usage\n\nUse QMD as a library in your own Node.js or Bun applications.\n\n#### Installation\n\n```sh\nnpm install @tobilu\u002Fqmd\n```\n\n#### Quick Start\n\n```typescript\nimport { createStore } from '@tobilu\u002Fqmd'\n\nconst store = await createStore({\n  dbPath: '.\u002Fmy-index.sqlite',\n  config: {\n    collections: {\n      docs: { path: '\u002Fpath\u002Fto\u002Fdocs', pattern: '**\u002F*.md' },\n    },\n  },\n})\n\nconst results = await store.search({ query: \"authentication flow\" })\nconsole.log(results.map(r => `${r.title} (${Math.round(r.score * 100)}%)`))\n\nawait store.close()\n```\n\n#### Store Creation\n\n`createStore()` accepts three modes:\n\n```typescript\nimport { createStore } from '@tobilu\u002Fqmd'\n\n\u002F\u002F 1. Inline config — no files needed besides the DB\nconst store = await createStore({\n  dbPath: '.\u002Findex.sqlite',\n  config: {\n    collections: {\n      docs: { path: '\u002Fpath\u002Fto\u002Fdocs', pattern: '**\u002F*.md' },\n      notes: { path: '\u002Fpath\u002Fto\u002Fnotes' },\n    },\n  },\n})\n\n\u002F\u002F 2. YAML config file — collections defined in a file\nconst store2 = await createStore({\n  dbPath: '.\u002Findex.sqlite',\n  configPath: '.\u002Fqmd.yml',\n})\n\n\u002F\u002F 3. DB-only — reopen a previously configured store\nconst store3 = await createStore({ dbPath: '.\u002Findex.sqlite' })\n```\n\n#### Search\n\nThe unified `search()` method handles both simple queries and pre-expanded structured queries:\n\n```typescript\n\u002F\u002F Simple query — auto-expanded via LLM, then BM25 + vector + reranking\nconst results = await store.search({ query: \"authentication flow\" })\n\n\u002F\u002F With options\nconst results2 = await store.search({\n  query: \"rate limiting\",\n  intent: \"API throttling and abuse prevention\",\n  collection: \"docs\",\n  limit: 5,\n  minScore: 0.3,\n  explain: true,\n})\n\n\u002F\u002F Pre-expanded queries — skip auto-expansion, control each sub-query\nconst results3 = await store.search({\n  queries: [\n    { type: 'lex', query: '\"connection pool\" timeout -redis' },\n    { type: 'vec', query: 'why do database connections time out under load' },\n  ],\n  collections: [\"docs\", \"notes\"],\n})\n\n\u002F\u002F Skip reranking for faster results\nconst fast = await store.search({ query: \"auth\", rerank: false })\n```\n\nFor direct backend access:\n\n```typescript\n\u002F\u002F BM25 keyword search (fast, no LLM)\nconst lexResults = await store.searchLex(\"auth middleware\", { limit: 10 })\n\n\u002F\u002F Vector similarity search (embedding model, no reranking)\nconst vecResults = await store.searchVector(\"how users log in\", { limit: 10 })\n\n\u002F\u002F Manual query expansion for full control\nconst expanded = await store.expandQuery(\"auth flow\", { intent: \"user login\" })\nconst results4 = await store.search({ queries: expanded })\n```\n\n#### Retrieval\n\n```typescript\n\u002F\u002F Get a document by path or docid\nconst doc = await store.get(\"docs\u002Freadme.md\")\nconst byId = await store.get(\"#abc123\")\n\nif (!(\"error\" in doc)) {\n  console.log(doc.title, doc.displayPath, doc.context)\n}\n\n\u002F\u002F Get document body with line range\nconst body = await store.getDocumentBody(\"docs\u002Freadme.md\", {\n  fromLine: 50,\n  maxLines: 100,\n})\n\n\u002F\u002F Batch retrieve by glob or comma-separated list\nconst { docs, errors } = await store.multiGet(\"docs\u002F**\u002F*.md\", {\n  maxBytes: 20480,\n})\n```\n\n#### Collections\n\n```typescript\n\u002F\u002F Add a collection\nawait store.addCollection(\"myapp\", {\n  path: \"\u002Fsrc\u002Fmyapp\",\n  pattern: \"**\u002F*.ts\",\n  ignore: [\"node_modules\u002F**\", \"*.test.ts\"],\n})\n\n\u002F\u002F List collections with document stats\nconst collections = await store.listCollections()\n\u002F\u002F => [{ name, pwd, glob_pattern, doc_count, active_count, last_modified, includeByDefault }]\n\n\u002F\u002F Get names of collections included in queries by default\nconst defaults = await store.getDefaultCollectionNames()\n\n\u002F\u002F Remove \u002F rename\nawait store.removeCollection(\"myapp\")\nawait store.renameCollection(\"old-name\", \"new-name\")\n```\n\n#### Context\n\nContext adds descriptive metadata that improves search relevance and is returned alongside results:\n\n```typescript\n\u002F\u002F Add context for a path within a collection\nawait store.addContext(\"docs\", \"\u002Fapi\", \"REST API reference documentation\")\n\n\u002F\u002F Set global context (applies to all collections)\nawait store.setGlobalContext(\"Internal engineering documentation\")\n\n\u002F\u002F List all contexts\nconst contexts = await store.listContexts()\n\u002F\u002F => [{ collection, path, context }]\n\n\u002F\u002F Remove context\nawait store.removeContext(\"docs\", \"\u002Fapi\")\nawait store.setGlobalContext(undefined)  \u002F\u002F clear global\n```\n\n#### Indexing\n\n```typescript\n\u002F\u002F Re-index collections by scanning the filesystem\nconst result = await store.update({\n  collections: [\"docs\"],  \u002F\u002F optional — defaults to all\n  onProgress: ({ collection, file, current, total }) => {\n    console.log(`[${collection}] ${current}\u002F${total} ${file}`)\n  },\n})\n\u002F\u002F => { collections, indexed, updated, unchanged, removed, needsEmbedding }\n\n\u002F\u002F Generate vector embeddings\nconst embedResult = await store.embed({\n  force: false,           \u002F\u002F true to re-embed everything\n  chunkStrategy: \"auto\",  \u002F\u002F \"regex\" (default) or \"auto\" (AST for code files)\n  onProgress: ({ current, total, collection }) => {\n    console.log(`Embedding ${current}\u002F${total}`)\n  },\n})\n```\n\n#### Types\n\nKey types exported for SDK consumers:\n\n```typescript\nimport type {\n  QMDStore,            \u002F\u002F The store interface\n  SearchOptions,       \u002F\u002F Options for search()\n  LexSearchOptions,    \u002F\u002F Options for searchLex()\n  VectorSearchOptions, \u002F\u002F Options for searchVector()\n  HybridQueryResult,   \u002F\u002F Search result with score, snippet, context\n  SearchResult,        \u002F\u002F Result from searchLex\u002FsearchVector\n  ExpandedQuery,       \u002F\u002F Typed sub-query { type: 'lex'|'vec'|'hyde', query }\n  DocumentResult,      \u002F\u002F Document metadata + body\n  DocumentNotFound,    \u002F\u002F Error with similarFiles suggestions\n  MultiGetResult,      \u002F\u002F Batch retrieval result\n  UpdateProgress,      \u002F\u002F Progress callback info for update()\n  UpdateResult,        \u002F\u002F Aggregated update result\n  EmbedProgress,       \u002F\u002F Progress callback info for embed()\n  EmbedResult,         \u002F\u002F Embedding result\n  StoreOptions,        \u002F\u002F createStore() options\n  CollectionConfig,    \u002F\u002F Inline config shape\n  IndexStatus,         \u002F\u002F From getStatus()\n  IndexHealthInfo,     \u002F\u002F From getIndexHealth()\n} from '@tobilu\u002Fqmd'\n```\n\nUtility exports:\n\n```typescript\nimport {\n  extractSnippet,              \u002F\u002F Extract a relevant snippet from text\n  addLineNumbers,              \u002F\u002F Add line numbers to text\n  DEFAULT_MULTI_GET_MAX_BYTES, \u002F\u002F Default max file size for multiGet (10KB)\n  Maintenance,                 \u002F\u002F Database maintenance operations\n} from '@tobilu\u002Fqmd'\n```\n\n#### Lifecycle\n\n```typescript\n\u002F\u002F Close the store — disposes LLM models and DB connection\nawait store.close()\n```\n\nThe SDK requires explicit `dbPath` — no defaults are assumed. This makes it safe to embed in any application without side effects.\n\n## Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│                         QMD Hybrid Search Pipeline                          │\n└─────────────────────────────────────────────────────────────────────────────┘\n\n                              ┌─────────────────┐\n                              │   User Query    │\n                              └────────┬────────┘\n                                       │\n                        ┌──────────────┴──────────────┐\n                        ▼                             ▼\n               ┌────────────────┐            ┌────────────────┐\n               │ Query Expansion│            │  Original Query│\n               │  (fine-tuned)  │            │   (×2 weight)  │\n               └───────┬────────┘            └───────┬────────┘\n                       │                             │\n                       │ 2 alternative queries       │\n                       └──────────────┬──────────────┘\n                                      │\n              ┌───────────────────────┼───────────────────────┐\n              ▼                       ▼                       ▼\n     ┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\n     │ Original Query  │     │ Expanded Query 1│     │ Expanded Query 2│\n     └────────┬────────┘     └────────┬────────┘     └────────┬────────┘\n              │                       │                       │\n      ┌───────┴───────┐       ┌───────┴───────┐       ┌───────┴───────┐\n      ▼               ▼       ▼               ▼       ▼               ▼\n  ┌───────┐       ┌───────┐ ┌───────┐     ┌───────┐ ┌───────┐     ┌───────┐\n  │ BM25  │       │Vector │ │ BM25  │     │Vector │ │ BM25  │     │Vector │\n  │(FTS5) │       │Search │ │(FTS5) │     │Search │ │(FTS5) │     │Search │\n  └───┬───┘       └───┬───┘ └───┬───┘     └───┬───┘ └───┬───┘     └───┬───┘\n      │               │         │             │         │             │\n      └───────┬───────┘         └──────┬──────┘         └──────┬──────┘\n              │                        │                       │\n              └────────────────────────┼───────────────────────┘\n                                       │\n                                       ▼\n                          ┌───────────────────────┐\n                          │   RRF Fusion + Bonus  │\n                          │  Original query: ×2   │\n                          │  Top-rank bonus: +0.05│\n                          │     Top 30 Kept       │\n                          └───────────┬───────────┘\n                                      │\n                                      ▼\n                          ┌───────────────────────┐\n                          │    LLM Re-ranking     │\n                          │  (qwen3-reranker)     │\n                          │  Yes\u002FNo + logprobs    │\n                          └───────────┬───────────┘\n                                      │\n                                      ▼\n                          ┌───────────────────────┐\n                          │  Position-Aware Blend │\n                          │  Top 1-3:  75% RRF    │\n                          │  Top 4-10: 60% RRF    │\n                          │  Top 11+:  40% RRF    │\n                          └───────────────────────┘\n```\n\n## Score Normalization & Fusion\n\n### Search Backends\n\n| Backend | Raw Score | Conversion | Range |\n|---------|-----------|------------|-------|\n| **FTS (BM25)** | SQLite FTS5 BM25 | `Math.abs(score)` | 0 to ~25+ |\n| **Vector** | Cosine distance | `1 \u002F (1 + distance)` | 0.0 to 1.0 |\n| **Reranker** | LLM 0-10 rating | `score \u002F 10` | 0.0 to 1.0 |\n\n### Fusion Strategy\n\nThe `query` command uses **Reciprocal Rank Fusion (RRF)** with position-aware blending:\n\n1. **Query Expansion**: Original query (×2 for weighting) + 1 LLM variation\n2. **Parallel Retrieval**: Each query searches both FTS and vector indexes\n3. **RRF Fusion**: Combine all result lists using `score = Σ(1\u002F(k+rank+1))` where k=60\n4. **Top-Rank Bonus**: Documents ranking #1 in any list get +0.05, #2-3 get +0.02\n5. **Top-K Selection**: Take top 30 candidates for reranking\n6. **Re-ranking**: LLM scores each document (yes\u002Fno with logprobs confidence)\n7. **Position-Aware Blending**:\n   - RRF rank 1-3: 75% retrieval, 25% reranker (preserves exact matches)\n   - RRF rank 4-10: 60% retrieval, 40% reranker\n   - RRF rank 11+: 40% retrieval, 60% reranker (trust reranker more)\n\n**Why this approach**: Pure RRF can dilute exact matches when expanded queries don't match. The top-rank bonus preserves documents that score #1 for the original query. Position-aware blending prevents the reranker from destroying high-confidence retrieval results.\n\n### Score Interpretation\n\n| Score | Meaning |\n|-------|---------|\n| 0.8 - 1.0 | Highly relevant |\n| 0.5 - 0.8 | Moderately relevant |\n| 0.2 - 0.5 | Somewhat relevant |\n| 0.0 - 0.2 | Low relevance |\n\n## Requirements\n\n### System Requirements\n\n- **Node.js** >= 22\n- **Bun** >= 1.0.0\n- **macOS**: Homebrew SQLite (for extension support)\n  ```sh\n  brew install sqlite\n  ```\n\n### GGUF Models (via node-llama-cpp)\n\nQMD uses three local GGUF models (auto-downloaded on first use):\n\n| Model | Purpose | Size |\n|-------|---------|------|\n| `embeddinggemma-300M-Q8_0` | Vector embeddings (default) | ~300MB |\n| `qwen3-reranker-0.6b-q8_0` | Re-ranking | ~640MB |\n| `qmd-query-expansion-1.7B-q4_k_m` | Query expansion (fine-tuned) | ~1.1GB |\n\nModels are downloaded from HuggingFace and cached in `~\u002F.cache\u002Fqmd\u002Fmodels\u002F`.\n\n### Custom Embedding Model\n\nOverride the default embedding model via the `QMD_EMBED_MODEL` environment variable.\nThis is useful for multilingual corpora (e.g. Chinese, Japanese, Korean) where\n`embeddinggemma-300M` has limited coverage.\n\n```sh\n# Use Qwen3-Embedding-0.6B for better multilingual (CJK) support\nexport QMD_EMBED_MODEL=\"hf:Qwen\u002FQwen3-Embedding-0.6B-GGUF\u002FQwen3-Embedding-0.6B-Q8_0.gguf\"\n\n# After changing the model, re-embed all collections:\nqmd embed -f\n```\n\nSupported model families:\n- **embeddinggemma** (default) — English-optimized, small footprint\n- **Qwen3-Embedding** — Multilingual (119 languages including CJK), MTEB top-ranked\n\n> **Note:** When switching embedding models, you must re-index with `qmd embed -f`\n> since vectors are not cross-compatible between models. The prompt format is\n> automatically adjusted for each model family.\n\n## Installation\n\n```sh\nnpm install -g @tobilu\u002Fqmd\n# or\nbun install -g @tobilu\u002Fqmd\n```\n\n### Development\n\n```sh\ngit clone https:\u002F\u002Fgithub.com\u002Ftobi\u002Fqmd\ncd qmd\nnpm install\nnpm link\n```\n\n## Usage\n\n### Collection Management\n\n```sh\n# Create a collection from current directory\nqmd collection add . --name myproject\n\n# Create a collection with explicit path and custom glob mask\nqmd collection add ~\u002FDocuments\u002Fnotes --name notes --mask \"**\u002F*.md\"\n\n# List all collections\nqmd collection list\n\n# Remove a collection\nqmd collection remove myproject\n\n# Rename a collection\nqmd collection rename myproject my-project\n\n# List files in a collection\nqmd ls notes\nqmd ls notes\u002Fsubfolder\n```\n\n### Generate Vector Embeddings\n\n```sh\n# Embed all indexed documents (900 tokens\u002Fchunk, 15% overlap)\nqmd embed\n\n# Force re-embed everything\nqmd embed -f\n\n# Enable AST-aware chunking for code files (TS, JS, Python, Go, Rust)\nqmd embed --chunk-strategy auto\n\n# Also works with query for consistent chunk selection\nqmd query \"auth flow\" --chunk-strategy auto\n```\n\n**AST-aware chunking** (`--chunk-strategy auto`) uses tree-sitter to chunk code\nfiles at function, class, and import boundaries instead of arbitrary text\npositions. This produces higher-quality chunks and better search results for\ncodebases. Markdown and other file types always use regex-based chunking\nregardless of strategy.\n\nThe default is `regex` (existing behavior). Use `--chunk-strategy auto` to\nopt in. Run `qmd status` to verify which grammars are available.\n\n> **Note:** Tree-sitter grammars are optional dependencies. If they are not\n> installed, `--chunk-strategy auto` falls back to regex-only chunking\n> automatically. Tested on both Node.js and Bun.\n\n### Context Management\n\nContext adds descriptive metadata to collections and paths, helping search understand your content.\n\n```sh\n# Add context to a collection (using qmd:\u002F\u002F virtual paths)\nqmd context add qmd:\u002F\u002Fnotes \"Personal notes and ideas\"\nqmd context add qmd:\u002F\u002Fdocs\u002Fapi \"API documentation\"\n\n# Add context from within a collection directory\ncd ~\u002Fnotes && qmd context add \"Personal notes and ideas\"\ncd ~\u002Fnotes\u002Fwork && qmd context add \"Work-related notes\"\n\n# Add global context (applies to all collections)\nqmd context add \u002F \"Knowledge base for my projects\"\n\n# List all contexts\nqmd context list\n\n# Remove context\nqmd context rm qmd:\u002F\u002Fnotes\u002Fold\n```\n\n### Search Commands\n\n```\n┌──────────────────────────────────────────────────────────────────┐\n│                        Search Modes                              │\n├──────────┬───────────────────────────────────────────────────────┤\n│ search   │ BM25 full-text search only                           │\n│ vsearch  │ Vector semantic search only                          │\n│ query    │ Hybrid: FTS + Vector + Query Expansion + Re-ranking  │\n└──────────┴───────────────────────────────────────────────────────┘\n```\n\n```sh\n# Full-text search (fast, keyword-based)\nqmd search \"authentication flow\"\n\n# Vector search (semantic similarity)\nqmd vsearch \"how to login\"\n\n# Hybrid search with re-ranking (best quality)\nqmd query \"user authentication\"\n```\n\n### Options\n\n```sh\n# Search options\n-n \u003Cnum>           # Number of results (default: 5, or 20 for --files\u002F--json)\n-c, --collection   # Restrict search to a specific collection\n--all              # Return all matches (use with --min-score to filter)\n--min-score \u003Cnum>  # Minimum score threshold (default: 0)\n--full             # Show full document content\n--line-numbers     # Add line numbers to output\n--explain          # Include retrieval score traces (query, JSON\u002FCLI output)\n--index \u003Cname>     # Use named index\n\n# Output formats (for search and multi-get)\n--files            # Output: docid,score,filepath,context\n--json             # JSON output with snippets\n--csv              # CSV output\n--md               # Markdown output\n--xml              # XML output\n\n# Get options\nqmd get \u003Cfile>[:line]  # Get document, optionally starting at line\n-l \u003Cnum>               # Maximum lines to return\n--from \u003Cnum>           # Start from line number\n\n# Multi-get options\n-l \u003Cnum>           # Maximum lines per file\n--max-bytes \u003Cnum>  # Skip files larger than N bytes (default: 10KB)\n```\n\n### Output Format\n\nDefault output is colorized CLI format (respects `NO_COLOR` env).\n\nWhen stdout is a TTY, result paths are emitted as clickable terminal hyperlinks (OSC 8). Clicking a path opens the file in your editor using an editor URI template.\n\nWhen stdout is not a TTY (for example piped to another command or redirected to a file), QMD emits plain text paths with no escape sequences.\n\nTTY example:\n\n```\ndocs\u002Fguide.md:42 #a1b2c3\nTitle: Software Craftsmanship\nContext: Work documentation\nScore: 93%\n\nThis section covers the **craftsmanship** of building\nquality software with attention to detail.\nSee also: engineering principles\n\n\nnotes\u002Fmeeting.md:15 #d4e5f6\nTitle: Q4 Planning\nContext: Personal notes and ideas\nScore: 67%\n\nDiscussion about code quality and craftsmanship\nin the development process.\n```\n\nConfigure the editor link target with `QMD_EDITOR_URI` (or `editor_uri` in config):\n\n```sh\n# VS Code (default)\nexport QMD_EDITOR_URI=\"vscode:\u002F\u002Ffile\u002F{path}:{line}:{col}\"\n\n# Cursor\nexport QMD_EDITOR_URI=\"cursor:\u002F\u002Ffile\u002F{path}:{line}:{col}\"\n\n# Zed\nexport QMD_EDITOR_URI=\"zed:\u002F\u002Ffile\u002F{path}:{line}:{col}\"\n\n# Sublime Text\nexport QMD_EDITOR_URI=\"subl:\u002F\u002Fopen?url=file:\u002F\u002F{path}&line={line}\"\n```\n\nTemplate placeholders:\n- `{path}` absolute filesystem path (URI-encoded)\n- `{line}` 1-based line number\n- `{col}` or `{column}` 1-based column number\n\n- **Path**: Collection-relative path (e.g., `docs\u002Fguide.md`)\n- **Docid**: Short hash identifier (e.g., `#a1b2c3`) - use with `qmd get #a1b2c3`\n- **Title**: Extracted from document (first heading or filename)\n- **Context**: Path context if configured via `qmd context add`\n- **Score**: Color-coded (green >70%, yellow >40%, dim otherwise)\n- **Snippet**: Context around match with query terms highlighted\n\n### Examples\n\n```sh\n# Get 10 results with minimum score 0.3\nqmd query -n 10 --min-score 0.3 \"API design patterns\"\n\n# Output as markdown for LLM context\nqmd search --md --full \"error handling\"\n\n# JSON output for scripting\nqmd query --json \"quarterly reports\"\n\n# Inspect how each result was scored (RRF + rerank blend)\nqmd query --json --explain \"quarterly reports\"\n\n# Use separate index for different knowledge base\nqmd --index work search \"quarterly reports\"\n```\n\n### Index Maintenance\n\n```sh\n# Show index status and collections with contexts\nqmd status\n\n# Re-index all collections\nqmd update\n\n# Re-index with git pull first (for remote repos)\nqmd update --pull\n\n# Get document by filepath (with fuzzy matching suggestions)\nqmd get notes\u002Fmeeting.md\n\n# Get document by docid (from search results)\nqmd get \"#abc123\"\n\n# Get document starting at line 50, max 100 lines\nqmd get notes\u002Fmeeting.md:50 -l 100\n\n# Get multiple documents by glob pattern\nqmd multi-get \"journals\u002F2025-05*.md\"\n\n# Get multiple documents by comma-separated list (supports docids)\nqmd multi-get \"doc1.md, doc2.md, #abc123\"\n\n# Limit multi-get to files under 20KB\nqmd multi-get \"docs\u002F*.md\" --max-bytes 20480\n\n# Output multi-get as JSON for agent processing\nqmd multi-get \"docs\u002F*.md\" --json\n\n# Clean up cache and orphaned data\nqmd cleanup\n```\n\n## Data Storage\n\nIndex stored in: `~\u002F.cache\u002Fqmd\u002Findex.sqlite`\n\n### Schema\n\n```sql\ncollections     -- Indexed directories with name and glob patterns\npath_contexts   -- Context descriptions by virtual path (qmd:\u002F\u002F...)\ndocuments       -- Markdown content with metadata and docid (6-char hash)\ndocuments_fts   -- FTS5 full-text index\ncontent_vectors -- Embedding chunks (hash, seq, pos, 900 tokens each)\nvectors_vec     -- sqlite-vec vector index (hash_seq key)\nllm_cache       -- Cached LLM responses (query expansion, rerank scores)\n```\n\n## Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `XDG_CACHE_HOME` | `~\u002F.cache` | Cache directory location |\n\n## How It Works\n\n### Indexing Flow\n\n```\nCollection ──► Glob Pattern ──► Markdown Files ──► Parse Title ──► Hash Content\n    │                                                   │              │\n    │                                                   │              ▼\n    │                                                   │         Generate docid\n    │                                                   │         (6-char hash)\n    │                                                   │              │\n    └──────────────────────────────────────────────────►└──► Store in SQLite\n                                                                       │\n                                                                       ▼\n                                                                  FTS5 Index\n```\n\n### Embedding Flow\n\nDocuments are chunked into ~900-token pieces with 15% overlap using smart boundary detection:\n\n```\nDocument ──► Smart Chunk (~900 tokens) ──► Format each chunk ──► node-llama-cpp ──► Store Vectors\n                │                           \"title | text\"        embedBatch()\n                │\n                └─► Chunks stored with:\n                    - hash: document hash\n                    - seq: chunk sequence (0, 1, 2...)\n                    - pos: character position in original\n```\n\n### Smart Chunking\n\nInstead of cutting at hard token boundaries, QMD uses a scoring algorithm to find natural markdown break points. This keeps semantic units (sections, paragraphs, code blocks) together.\n\n**Break Point Scores:**\n\n| Pattern | Score | Description |\n|---------|-------|-------------|\n| `# Heading` | 100 | H1 - major section |\n| `## Heading` | 90 | H2 - subsection |\n| `### Heading` | 80 | H3 |\n| `#### Heading` | 70 | H4 |\n| `##### Heading` | 60 | H5 |\n| `###### Heading` | 50 | H6 |\n| ` ``` ` | 80 | Code block boundary |\n| `---` \u002F `***` | 60 | Horizontal rule |\n| Blank line | 20 | Paragraph boundary |\n| `- item` \u002F `1. item` | 5 | List item |\n| Line break | 1 | Minimal break |\n\n**Algorithm:**\n\n1. Scan document for all break points with scores\n2. When approaching the 900-token target, search a 200-token window before the cutoff\n3. Score each break point: `finalScore = baseScore × (1 - (distance\u002Fwindow)² × 0.7)`\n4. Cut at the highest-scoring break point\n\nThe squared distance decay means a heading 200 tokens back (score ~30) still beats a simple line break at the target (score 1), but a closer heading wins over a distant one.\n\n**Code Fence Protection:** Break points inside code blocks are ignored—code stays together. If a code block exceeds the chunk size, it's kept whole when possible.\n\n**AST-Aware Chunking (Code Files):**\n\nFor supported code files, QMD also parses the source with [tree-sitter](https:\u002F\u002Ftree-sitter.github.io\u002F) and adds AST-derived break points that are merged with the regex scores above:\n\n| AST Node | Score | Languages |\n|----------|-------|-----------|\n| Class \u002F interface \u002F struct \u002F impl \u002F trait | 100 | All |\n| Function \u002F method | 90 | All |\n| Type alias \u002F enum | 80 | All |\n| Import \u002F use declaration | 60 | All |\n\nSupported for `.ts`, `.tsx`, `.js`, `.jsx`, `.py`, `.go`, and `.rs` files. Enable with `--chunk-strategy auto`. Markdown and other file types always use regex chunking.\n\n### Query Flow (Hybrid)\n\n```\nQuery ──► LLM Expansion ──► [Original, Variant 1, Variant 2]\n                │\n      ┌─────────┴─────────┐\n      ▼                   ▼\n   For each query:     FTS (BM25)\n      │                   │\n      ▼                   ▼\n   Vector Search      Ranked List\n      │\n      ▼\n   Ranked List\n      │\n      └─────────┬─────────┘\n                ▼\n         RRF Fusion (k=60)\n         Original query ×2 weight\n         Top-rank bonus: +0.05\u002F#1, +0.02\u002F#2-3\n                │\n                ▼\n         Top 30 candidates\n                │\n                ▼\n         LLM Re-ranking\n         (yes\u002Fno + logprob confidence)\n                │\n                ▼\n         Position-Aware Blend\n         Rank 1-3:  75% RRF \u002F 25% reranker\n         Rank 4-10: 60% RRF \u002F 40% reranker\n         Rank 11+:  40% RRF \u002F 60% reranker\n                │\n                ▼\n         Final Results\n```\n\n## Model Configuration\n\nModels are configured in `src\u002Fllm.ts` as HuggingFace URIs:\n\n```typescript\nconst DEFAULT_EMBED_MODEL = \"hf:ggml-org\u002Fembeddinggemma-300M-GGUF\u002Fembeddinggemma-300M-Q8_0.gguf\";\nconst DEFAULT_RERANK_MODEL = \"hf:ggml-org\u002FQwen3-Reranker-0.6B-Q8_0-GGUF\u002Fqwen3-reranker-0.6b-q8_0.gguf\";\nconst DEFAULT_GENERATE_MODEL = \"hf:tobil\u002Fqmd-query-expansion-1.7B-gguf\u002Fqmd-query-expansion-1.7B-q4_k_m.gguf\";\n```\n\n### EmbeddingGemma Prompt Format\n\n```\n\u002F\u002F For queries\n\"task: search result | query: {query}\"\n\n\u002F\u002F For documents\n\"title: {title} | text: {content}\"\n```\n\n### Qwen3-Reranker\n\nUses node-llama-cpp's `createRankingContext()` and `rankAndSort()` API for cross-encoder reranking. Returns documents sorted by relevance score (0.0 - 1.0).\n\n### Qwen3 (Query Expansion)\n\nUsed for generating query variations via `LlamaChatSession`.\n\n## License\n\nMIT\n","QMD 是一个本地运行的文档搜索引擎，支持对个人笔记、会议记录、文档和知识库进行索引和搜索。它结合了BM25全文检索、向量语义搜索以及基于LLM的重排序技术，所有处理都在本地完成，确保了数据隐私与安全性。QMD特别适合需要快速查找信息但又希望保持数据私密性的场景，如个人知识管理、团队协作中的文档查询等。此外，其设计考虑到了与AI代理的集成需求，提供了JSON格式输出及MCP服务器功能以支持更复杂的自动化流程。",2,"2026-06-11 02:55:24","top_language"]