[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-999":3},{"id":4,"name":5,"fullName":6,"owner":5,"repo":5,"description":7,"homepage":8,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":15,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":17,"rankGlobal":9,"rankLanguage":9,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":9,"pushedAt":9,"updatedAt":33,"readmeContent":34,"aiSummary":35,"trendingCount":15,"starSnapshotCount":15,"syncStatus":36,"lastSyncTime":37,"discoverSource":38},999,"kiwifs","kiwifs\u002Fkiwifs","Markdown filesystem for agents and teams.","https:\u002F\u002Fkiwifs.mintlify.app",null,"Go",686,155,18,6,0,349,10.58,"Other",false,"main",true,[23,24,25,26,27,28,29,30,31,32],"ai-agents","filesystem","golang","knowledge-base","knowledge-graph","llm-wiki","markdown","obsidian","self-hosted","wiki","2026-06-12 02:00:21","\u003Cp align=\"center\">\n  \u003Cimg src=\"kiwifs.png\" alt=\"KiwiFS\" width=\"200\" \u002F>\n\u003C\u002Fp>\n\n\u003Ch1 align=\"center\">KiwiFS\u003C\u002Fh1>\n\n\u003Cp align=\"center\">\n  \u003Cstrong>Knowledge filesystem agents can write to, search, query, and trust.\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  Agents write with \u003Ccode>cat\u003C\u002Fcode>. Humans read in a wiki. Git versions everything. One binary, zero config.\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fkiwifs\u002Fkiwifs\u002Factions\u002Fworkflows\u002Fci.yml\">\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fkiwifs\u002Fkiwifs\u002Factions\u002Fworkflows\u002Fci.yml\u002Fbadge.svg\" alt=\"CI\">\u003C\u002Fa>\n  \u003Ca href=\"LICENSE\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-BSL--1.1-blue\" alt=\"License: BSL 1.1\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fkiwifs\u002Fkiwifs\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fgo-1.25-00ADD8?logo=go&logoColor=white\" alt=\"Go 1.25\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fkiwifs\u002Fkiwifs\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fsingle_binary-yes-green\" alt=\"Single Binary\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fkiwifs\u002Fkiwifs\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPRs-welcome-brightgreen\" alt=\"PRs Welcome\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fkiwifs.mintlify.app\">Docs\u003C\u002Fa> · \u003Ca href=\"FAQ.md\">FAQ\u003C\u002Fa> · \u003Ca href=\"ROADMAP.md\">Roadmap\u003C\u002Fa> · \u003Ca href=\"CONTRIBUTING.md\">Contributing\u003C\u002Fa>\n\u003C\u002Fp>\n\n```bash\ncurl -fsSL https:\u002F\u002Fraw.githubusercontent.com\u002Fkiwifs\u002Fkiwifs\u002Fmain\u002Finstall.sh | sh\nkiwifs init --root .\u002Fknowledge && kiwifs serve --root .\u002Fknowledge\n# Open http:\u002F\u002Flocalhost:3333\n```\n\n**Or with Docker:**\n\n```bash\ndocker run -p 3333:3333 -v .\u002Fknowledge:\u002Fdata ameliaanhlam\u002Fkiwifs\n```\n\n---\n\n## The problem\n\nKnowledge filesystems for agents are an emerging primitive — but files are just files. Or are they?\n\nCurrent solutions fall into one of these camps:\n\n- **Database tables pretending to be files** — no search, no versioning, no human interface. You get `read()` and `write()`, nothing else.\n- **Read-only retrieval layers** — agents can search but can't write. The filesystem is a one-way mirror.\n- **Flat markdown logs** — no structure, no queries, no importance scoring. The naive approach everyone outgrows.\n- **Ephemeral sandboxes** — agent dies, files die. No persistence across sessions.\n- **Proprietary SaaS** — locked to a vendor's ecosystem. Can't self-host, can't extend.\n\nA real knowledge filesystem needs to be all of these at once:\n\n- **Writable** — agents write with `cat`, `echo`, `curl`, or MCP tools. Not read-only, not API-only.\n- **Searchable** — full-text (BM25) and semantic (vector) over the same files.\n- **Queryable** — structured queries over typed metadata, not just keyword matching.\n- **Trustworthy** — every write is a Git commit. Immutable audit trail. Crash recovery. Blame.\n- **Human-readable** — a web UI with wiki links, backlinks, and graph view. Not pgAdmin.\n\nKiwiFS is all five.\n\n```\nAGENT                              HUMAN\n─────                              ─────\ncat \u002Fkiwi\u002Fpages\u002Fauth.md            Web UI (like Obsidian Publish\ngrep -r \"timeout\" \u002Fkiwi\u002F             + Notion's block editor)\necho \"# Report\" > \u002Fkiwi\u002Fr.md      wiki links, graph view, search\n\n       ↕                                ↕\n     ┌──────────────────────────────────────┐\n     │         Markdown files in folders    │\n     │         (the single source of truth) │\n     └──────────────────────────────────────┘\n       ↕                ↕              ↕\n    Git versioning   FTS5 + vector   SSE events\n    (audit trail)    (search index)  (live updates)\n```\n\n## Why files, not a database\n\nThis is the core design decision. Every other choice follows from it.\n\n**Files are the only format that is simultaneously human-readable, agent-native, and tool-agnostic.** `cat file.md` works in every shell, every container, every sandbox, every CI pipeline. No driver. No connection string. No SDK. The agent doesn't need to learn your API — it already knows the filesystem from training data.\n\nA Postgres table is invisible to both agents and humans without custom tooling. A JSON blob requires parsing. A proprietary format requires a client library. Markdown files require nothing.\n\nBut raw files alone aren't enough. You need versioning, search, concurrency control, a web UI. KiwiFS layers database-like guarantees **on top of** files:\n\n| Need | How KiwiFS solves it |\n|---|---|\n| **Versioning** | Git — every write is an atomic commit. Crash recovery, blame, diff, point-in-time restore. |\n| **Search** | SQLite FTS5 (BM25 ranked) + pluggable vector search. Rebuildable from files anytime. |\n| **Concurrency** | Optimistic locking via ETags (git blob hash). Standard HTTP `If-Match` \u002F `409 Conflict`. |\n| **Structured queries** | Frontmatter → SQLite `file_meta` table. Query by field, sort, filter. |\n| **Audit trail** | Git commit log. SHA-1 hash chain. Tamper = broken chain. |\n| **Real-time sync** | SSE broadcast on every write\u002Fdelete. UI updates live. |\n\nThe files are the truth. Everything else is a derivative index you can rebuild.\n\n> **\"I already have this in Postgres.\"**\n>\n> Postgres stores your data. KiwiFS makes it *accessible* — to agents via `cat`\u002F`grep`\u002FNFS\u002FS3, to humans via a web UI with wiki links and graph view, to auditors via `git blame`. If your agent's knowledge lives in Postgres, your humans need a custom UI to review it, your agents need SQL to query it, and you have no audit trail. KiwiFS gives you all three interfaces over the same files.\n\n## The LLM Wiki pattern\n\nKiwiFS implements [Karpathy's LLM Wiki](https:\u002F\u002Fgist.github.com\u002Fkarpathy\u002F442a6bf555914893e9891c11519de94f) as production infrastructure. The pattern: raw sources in, compiled wiki out, agent maintains it over time.\n\n```bash\nkiwifs init --template knowledge\n```\n\n```\nknowledge\u002F\n├── SCHEMA.md          # Structure and frontmatter conventions\n├── index.md           # Auto-maintained table of contents\n├── log.md             # Append-only chronological record\n├── pages\u002F             # Durable knowledge — one page per concept, entity, or topic\n├── episodes\u002F          # Per-session episodic notes (consolidate into pages over time)\n└── .kiwi\u002F\n    └── playbook.md    # Agent-readable operation guide (MCP tool sequences)\n```\n\nEvery template ships with two agent-facing documents:\n- **`SCHEMA.md`** — structure, directory layout, frontmatter field tables\n- **`.kiwi\u002Fplaybook.md`** — step-by-step MCP tool sequences for each operation\n\nThe agent calls `kiwi_context` on connect to receive both documents plus the current index in one call. Operations from the playbook:\n\n- **Ingest** — process a new source, create\u002Fupdate pages, update index + log\n- **Query** — search the wiki to answer a question\n- **Remember** — save episodic observations during a session\n- **Consolidate** — merge episodes into durable pages\n- **Lint** — audit for orphan pages, broken links, stale content\n\nOther templates: `wiki`, `runbook`, `research`, or start blank with `kiwifs init`.\n\n---\n\n## Features\n\n### Web UI\n\nEmbedded in the binary via `go:embed`. No separate frontend deploy, no Node runtime. Obsidian's knowledge features (wiki links, backlinks, graph view) with a Notion-style block editor.\n\n- **WYSIWYG editor** — block-based (BlockNote), drag handles, 15+ block types, slash commands\n- **`[[Wiki links]]` + backlinks** — type `[[auth]]`, resolves to `pages\u002Fauthentication.md`. Backlinks panel shows \"linked from 3 pages.\" This is Obsidian's core feature — notes are connected, not isolated.\n- **Knowledge graph** — visual map of all pages and their connections (Sigma.js + ForceAtlas2). Same organic clustering as Obsidian's graph view.\n- **Cmd+K search** — full-text with highlighted matches\n- **Breadcrumbs** — `Knowledge > Pages > Authentication`\n- **Table of contents** — auto-generated, sticky sidebar, scroll tracking\n- **Inline comments** — select text, add annotation. Stored in `.kiwi\u002Fcomments\u002F`, not in the markdown\n- **Dark mode** — toggle a CSS class\n- **Themeable** — CSS variables, Tailwind-based, drop into any shadcn\u002Fui project\n\nBuilt on shadcn\u002Fui + Radix. Accessible. Beautiful by default. Fully customizable.\n\n### Agent interface\n\nWhen your agent has a real filesystem mount:\n\n```bash\ncat \u002Fkiwi\u002Fpages\u002Fauthentication.md\ngrep -r \"timeout\" \u002Fkiwi\u002F\nls \u002Fkiwi\u002Fpages\u002F\necho \"# New finding\" > \u002Fkiwi\u002Fpages\u002Ffinding-042.md\n```\n\nWhen a real mount isn't available, agents use the REST API or MCP tools instead.\n\n### MCP (Model Context Protocol)\n\n```bash\nkiwifs mcp --root ~\u002Fknowledge          # stdio (default), in-process, no server needed\nkiwifs mcp --root ~\u002Fknowledge --http   # Streamable HTTP transport on :8080\nkiwifs mcp --remote http:\u002F\u002Fhost:3333   # proxy to a running KiwiFS server\n```\n\n21 tools: `kiwi_context`, `kiwi_read`, `kiwi_write`, `kiwi_search`, `kiwi_tree`, `kiwi_query_meta`, `kiwi_delete`, `kiwi_bulk_write`, `kiwi_rename`, `kiwi_query`, `kiwi_aggregate`, `kiwi_import`, `kiwi_export`, `kiwi_analytics`, `kiwi_memory_report`, `kiwi_view_refresh`, `kiwi_health_check`, `kiwi_changes`, `kiwi_append`, `kiwi_search_semantic`, `kiwi_backlinks`. Plus resources (`kiwi:\u002F\u002Fschema`, `kiwi:\u002F\u002Ffile\u002F{path}`, `kiwi:\u002F\u002Ftree\u002F{path}`).\n\n**Claude Desktop \u002F Cursor:**\n```json\n{\n  \"mcpServers\": {\n    \"kiwifs\": {\n      \"command\": \"kiwifs\",\n      \"args\": [\"mcp\", \"--root\", \"\u002Fpath\u002Fto\u002Fknowledge\"]\n    }\n  }\n}\n```\n\n### Search (three tiers)\n\n```bash\nkiwifs serve --search grep      # Tier 1: zero deps, exact match\nkiwifs serve --search sqlite    # Tier 2: SQLite FTS5, BM25 ranked (default)\nkiwifs serve                    # Tier 2 + vector: enable [search.vector] in config.toml\n```\n\n```\nGET \u002Fapi\u002Fkiwi\u002Fsearch?q=payment+timeout           → BM25 ranked results\nGET \u002Fapi\u002Fkiwi\u002Fsearch?q=\"connection reset\" AND ws  → boolean + phrase search\nPOST \u002Fapi\u002Fkiwi\u002Fsearch\u002Fsemantic                    → vector similarity search\n```\n\nVector search is pluggable — mix any embedder with any vector store:\n\n| Embedder | Vector Store |\n|---|---|\n| OpenAI, Ollama, Cohere, Vertex AI, Bedrock, custom HTTP | sqlite-vec (default), Qdrant, pgvector, Pinecone, Weaviate, Milvus |\n\nDefault (sqlite-vec + OpenAI) needs one env var and zero infrastructure. For fully offline setups: Ollama + sqlite-vec, everything runs locally.\n\n### Git versioning\n\nEvery write is an atomic git commit. Users never see Git — the API abstracts it.\n\n```\nGET \u002Fapi\u002Fkiwi\u002Fversions?path=pages\u002Fauth.md      → commit history\nGET \u002Fapi\u002Fkiwi\u002Fdiff?path=auth.md&from=a1b&to=c3d → unified diff\nGET \u002Fapi\u002Fkiwi\u002Fblame?path=pages\u002Fauth.md          → per-line attribution\n```\n\nWhat Git gives you for free: crash recovery, immutable audit trail (SHA-1 hash chain), point-in-time restore, tamper detection, replication via `git push`.\n\n### Access protocols\n\nEvery protocol flows through the same storage layer. Every write — regardless of how it enters — gets a git commit, a search index update, and an SSE broadcast.\n\n```\n┌─────────────────────────────────────────┐\n│              KiwiFS Server              │\n│                                         │\n│  REST API  │  NFS   │  S3    │ WebDAV   │\n│   :3333    │ :2049  │ :3334  │  :3335   │\n│            └────┬───┘        │          │\n│       ──────────┴────────────┘          │\n│    Storage → Git → Index → SSE          │\n└─────────────────────────────────────────┘\n```\n\n| Protocol | Use case | Example |\n|---|---|---|\n| **REST API** | Web frontend, scripts | `curl localhost:3333\u002Fapi\u002Fkiwi\u002Ffile?path=index.md` |\n| **MCP** | AI agents (Claude, Cursor, custom) | `kiwifs mcp --root ~\u002Fknowledge` |\n| **NFS** | Docker, Kubernetes (native mount, no FUSE, no privileged) | `docker run --mount type=nfs,...` |\n| **S3** | Backup, data pipelines, any tool that \"supports S3\" | `aws s3 sync s3:\u002F\u002Fknowledge\u002F \u002Fbackup\u002F` |\n| **WebDAV** | Windows mapped drives, legacy tools | Map Network Drive in Explorer |\n| **FUSE** | Developer workstations, remote mount as local folder | `kiwifs mount --remote http:\u002F\u002Fserver:3333 ~\u002Fkiwi` |\n\n### Structured metadata\n\nFrontmatter from every markdown file is mirrored into a SQLite table. Query it:\n\n```\nGET \u002Fapi\u002Fkiwi\u002Fmeta?where=$.status=published&where=$.priority=high&sort=$.updated&order=desc\n```\n\n### DataView Query Language (DQL)\n\nA query language for your knowledge base — think Obsidian Dataview, but server-side:\n\n```bash\nkiwifs query 'TABLE title, status, priority FROM \"pages\" WHERE status = \"draft\" SORT priority DESC'\n```\n\n```\nGET \u002Fapi\u002Fkiwi\u002Fquery?q=TABLE title, status FROM \"pages\" WHERE priority = \"high\"\n```\n\nSupports `TABLE`, `LIST`, `COUNT`, `DISTINCT` queries with `WHERE`, `SORT`, `GROUP BY`, `FLATTEN`, and implicit fields (`_path`, `_updated`, `_size`). Expressions, functions, and boolean logic all work.\n\n**Computed views** — markdown files with `kiwi-view: true` in frontmatter auto-refresh their body from a DQL query:\n\n```bash\nkiwifs view create --query 'TABLE title, status FROM \"pages\"' --output views\u002Fpages.md\nkiwifs view refresh   # re-run all view queries\n```\n\n### Aggregation\n\nSQL-style aggregates over frontmatter fields:\n\n```bash\nkiwifs aggregate --group status --calc count,avg:priority\n```\n\n```\nGET \u002Fapi\u002Fkiwi\u002Fquery\u002Faggregate?group_by=status&calc=count,avg:priority\n```\n\nFunctions: `count`, `avg`, `sum`, `min`, `max`. Optional `--where` filters and `--path-prefix` scoping.\n\n### Computed frontmatter\n\nDefine expressions in config that are evaluated at index time and stored as virtual frontmatter fields:\n\n```toml\n# .kiwi\u002Fconfig.toml\n[dataview]\ncomputed_fields.age_days = \"days_since(updated)\"\ncomputed_fields.is_long = \"len(body) > 5000\"\ncomputed_fields.priority_score = \"priority * 10 + len(tags)\"\n```\n\nThese fields appear in DQL queries and meta API responses alongside real frontmatter.\n\n### Data import\n\nBulk-import data from 18 sources into your knowledge base. Each row becomes a markdown file with structured frontmatter:\n\n```bash\nkiwifs import --from postgres --dsn \"postgres:\u002F\u002F...\" --table users --root .\u002Fknowledge\nkiwifs import --from csv --path data.csv --root .\u002Fknowledge\nkiwifs import --from json --url https:\u002F\u002Fapi.example.com\u002Fdata --root .\u002Fknowledge\n```\n\n| Category | Sources |\n|---|---|\n| **Databases** | PostgreSQL, MySQL, SQLite, MongoDB, DynamoDB, Redis, Elasticsearch |\n| **Files** | CSV, JSON, JSONL, YAML, Excel |\n| **SaaS** | Notion, Airtable, Google Sheets, Confluence |\n| **Knowledge** | Obsidian vaults, Firebase\u002FFirestore |\n\nFeatures: idempotent upserts (re-importing skips unchanged rows), `--dry-run`, `--columns` filtering, `--primary-key` control, `_source` \u002F `_source_id` tracking in frontmatter.\n\n### Data export\n\nExport your knowledge base to machine-readable formats for ML pipelines, backups, or analysis:\n\n```bash\nkiwifs export --format jsonl --output knowledge.jsonl\nkiwifs export --format csv --include-embeddings --output dataset.csv\n```\n\n```\nGET \u002Fapi\u002Fkiwi\u002Fexport?format=jsonl&include_content=true&include_embeddings=true\n```\n\nFormats: JSONL, CSV. Optional: `--include-content` (full markdown body), `--include-links` (wiki link graph), `--include-embeddings` (vector embeddings), `--columns` filtering. Writes a `.schema.json` sidecar when exporting embeddings.\n\n### Analytics dashboard\n\nKnowledge health metrics at a glance:\n\n```bash\nkiwifs analytics                     # text summary\nkiwifs analytics --format json       # structured output\n```\n\n```\nGET \u002Fapi\u002Fkiwi\u002Fanalytics → { total_pages, stale_pages, orphans, broken_links, ... }\nGET \u002Fapi\u002Fkiwi\u002Fhealth-check?path=pages\u002Fauth.md    → per-page health\n```\n\nReports: total pages, stale page count + paths, orphan pages, broken links, empty pages, pages without frontmatter, link coverage percentage, recently updated pages.\n\n### Provenance tracking\n\nKnow which agent run produced which knowledge:\n\n```bash\ncurl -X PUT localhost:3333\u002Fapi\u002Fkiwi\u002Ffile?path=report.md \\\n  -H \"X-Actor: agent:exec_abc\" \\\n  -H \"X-Provenance: run:run-249\" \\\n  -d \"# Run 249 Report...\"\n```\n\nKiwiFS injects `derived-from` into the frontmatter automatically. Query later: \"show me every page produced by run-249.\"\n\n### Episodic and central memory\n\nModel **per-run \u002F episodic** notes separately from **semantic** concept pages, and use frontmatter `merged-from` to record which episodes were consolidated into a page. Run `kiwifs memory report` to list episodic files that are not yet referenced from any `merged-from` (for CI, dashboards, and merge jobs). Conventions, `[memory]` config, and a Go API live in [docs\u002FMEMORY.md](docs\u002FMEMORY.md).\n\n### Multi-space\n\nOne server, multiple independent knowledge bases:\n\n```\nGET \u002Fapi\u002Fkiwi\u002F{space}\u002Ftree\nGET \u002Fapi\u002Fkiwi\u002F{space}\u002Ffile?path=...\n```\n\nEach space has its own root directory, git repo, search index. Spaces map to NFS exports and S3 buckets.\n\n### Real-time events\n\n```\nGET \u002Fapi\u002Fkiwi\u002Fevents → SSE stream\n\nevent: write\ndata: {\"path\":\"pages\u002Ffinding-042.md\",\"actor\":\"agent:exec_abc\"}\n```\n\nUI updates live when knowledge changes. No polling.\n\n### Permalinks\n\nSet `public_url` in config and every API response includes stable, shareable URLs:\n\n```\nhttps:\u002F\u002Fwiki.mycompany.com\u002Fpage\u002Fpages\u002Fauthentication.md\n```\n\n- **SPA deep linking** — `\u002Fpage\u002F{path}` routes via HTML5 history (no `#` fragments)\n- **Wiki link resolution** — `[[auth]]` resolves to the full permalink URL for external contexts (Slack, PR comments, exports)\n- **X-Permalink header** — every file read returns the permalink in the response header\n- **`KIWI_PUBLIC_URL` env var** — override config for Docker\u002FCI without editing TOML\n\n### Knowledge health\n\nBuilt-in janitor scans your knowledge base for quality issues:\n\n- **Stale page detection** — pages not reviewed within a configurable window (default 90 days)\n- **Contradiction finder** — pages with conflicting claims on the same topic\n- **Trust-ranked search** — pages marked `status: verified` or `source-of-truth: true` rank higher\n- **Scheduled scans** — background janitor runs on a configurable interval (default 24h)\n- **Share links** — password-protected public access to specific pages (bcrypt-hashed)\n\n### CLI\n\nEvery feature is accessible via `kiwifs \u003Ccommand>`:\n\n| Command | What it does |\n|---|---|\n| `kiwifs serve` | Start the server (REST API + web UI + optional NFS\u002FS3\u002FWebDAV) |\n| `kiwifs init` | Scaffold a knowledge base from a template (`knowledge`, `wiki`, `runbook`, `research`, or `blank`) |\n| `kiwifs mcp` | Start a Model Context Protocol server (for Claude, Cursor, etc.) |\n| `kiwifs query` | Run a DQL query against the local index |\n| `kiwifs import` | Bulk-import from 18 data sources (Postgres, CSV, Notion, etc.) |\n| `kiwifs export` | Export knowledge base to JSONL or CSV |\n| `kiwifs aggregate` | Run SQL aggregates (count, avg, sum, min, max) over frontmatter |\n| `kiwifs analytics` | Knowledge health dashboard (stale, orphans, broken links) |\n| `kiwifs view` | Manage computed views (create, refresh, list) |\n| `kiwifs mount` | FUSE-mount a remote KiwiFS server as a local folder |\n| `kiwifs reindex` | Rebuild search indexes from files (FTS5 + vector + metadata) |\n| `kiwifs lint` | Validate knowledge base (orphan pages, broken links, missing frontmatter) |\n| `kiwifs backup` | Push to a git remote for off-site backup |\n| `kiwifs restore` | Clone from a git remote and rebuild indexes |\n| `kiwifs janitor` | Run a knowledge health scan (stale pages, contradictions, orphans) |\n| `kiwifs memory` | Report episodic vs `merged-from` coverage (see [docs\u002FMEMORY.md](docs\u002FMEMORY.md)) |\n\nAll commands support `--help` for full flag reference.\n\n---\n\n## Quickstart\n\n### 1. Install\n\n```bash\n# One-line install (macOS \u002F Linux)\ncurl -fsSL https:\u002F\u002Fraw.githubusercontent.com\u002Fkiwifs\u002Fkiwifs\u002Fmain\u002Finstall.sh | sh\n```\n\nOr run with Docker:\n\n```bash\ndocker run -v .\u002Fknowledge:\u002Fdata -p 3333:3333 ameliaanhlam\u002Fkiwifs\n```\n\nOr build from source (requires Go 1.25+ and Node.js 20+):\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fkiwifs\u002Fkiwifs.git && cd kiwifs\ncd ui && npm install && npm run build && cd ..\ngo build -o kiwifs .\n```\n\n### 2. Initialize\n\n```bash\nkiwifs init --template knowledge --root .\u002Fknowledge\n# Creates SCHEMA.md, index.md, log.md, pages\u002F, episodes\u002F, .kiwi\u002Fplaybook.md\n```\n\n### 3. Serve\n\n```bash\nkiwifs serve --root .\u002Fknowledge\n# REST API on :3333, web UI at http:\u002F\u002Flocalhost:3333\n```\n\n### 4. Write from an agent\n\n```bash\n# Via filesystem (if mounted):\necho \"# Authentication\\n\\nOAuth2 + JWT...\" > .\u002Fknowledge\u002Fpages\u002Fauth.md\n\n# Via API:\ncurl -X PUT 'localhost:3333\u002Fapi\u002Fkiwi\u002Ffile?path=pages\u002Fauth.md' \\\n  -H \"X-Actor: my-agent\" \\\n  -d \"# Authentication\\n\\nOAuth2 + JWT...\"\n```\n\n### 5. Browse in the web UI\n\nOpen `http:\u002F\u002Flocalhost:3333`. See `pages\u002Fauth.md` rendered as a styled page with wiki links, backlinks, and table of contents.\n\n---\n\n## Configuration\n\n```toml\n# .kiwi\u002Fconfig.toml\n\n[server]\nport = 3333\nhost = \"0.0.0.0\"\npublic_url = \"https:\u002F\u002Fwiki.mycompany.com\"  # enables permalinks\n\n[storage]\nroot = \"\u002Fdata\u002Fknowledge\"\n\n[search]\nengine = \"sqlite\"                # grep | sqlite\n\n[search.vector]\nenabled = true\n# Optional: lower for small CPUs\u002Flocal embedders, default is 5.\nworker_count = 1\n\n[search.vector.embedder]\nprovider = \"ollama\"              # openai | ollama | cohere | bedrock | vertex | http\nmodel = \"nomic-embed-text\"\n# Optional for Ollama: Go duration string, default is 30s.\ntimeout = \"120s\"\n\n[search.vector.store]\nprovider = \"sqlite-vec\"          # sqlite-vec | qdrant | pgvector | pinecone | weaviate | milvus\n\n[versioning]\nstrategy = \"git\"                 # git | cow | none\n\n[memory]\n# episodes_path_prefix = \"episodes\u002F\"   # optional; default episodes\u002F\n\n[auth]\ntype = \"none\"                    # none | apikey | perspace | oidc\n```\n\nCLI flags override config: `kiwifs serve --port 4000 --search sqlite --versioning git`.\n\n---\n\n## Deployment\n\n### Docker\n\n```bash\n# Pull from Docker Hub\ndocker run -v .\u002Fknowledge:\u002Fdata -p 3333:3333 ameliaanhlam\u002Fkiwifs\n\n# Or build locally\ndocker build -t kiwifs .\ndocker run -v .\u002Fknowledge:\u002Fdata -p 3333:3333 kiwifs\n```\n\n### Docker Compose (with vector search)\n\nFor vector search with pgvector, add a pgvector sidecar alongside KiwiFS. Configure `[search.vector.store] provider = \"pgvector\"` in `.kiwi\u002Fconfig.toml`.\n\n### Embedded in your app (Go library)\n\n```go\nimport \"github.com\u002Fkiwifs\u002Fkiwifs\u002Fpkg\u002Fkiwi\"\n\nsrv, err := kiwi.New(\"\u002Fdata\u002Fknowledge\", kiwi.WithSearch(\"sqlite\"))\nif err != nil { log.Fatal(err) }\ndefer srv.Close()\n\n\u002F\u002F Mount as an HTTP handler alongside your own routes\nmux.Handle(\"\u002Fknowledge\u002F\", http.StripPrefix(\"\u002Fknowledge\", srv.Handler()))\n\n\u002F\u002F Or run standalone\nlog.Fatal(srv.ListenAndServe(\":3333\"))\n```\n\n### With NFS (Docker \u002F Kubernetes)\n\n```bash\nkiwifs serve --root \u002Fdata\u002Fknowledge --nfs --nfs-port 2049\n```\n\n```yaml\n# Kubernetes PersistentVolume\napiVersion: v1\nkind: PersistentVolume\nspec:\n  nfs:\n    server: kiwifs.internal\n    path: \u002F\n```\n\n---\n\n## REST API\n\n```\nGET    \u002Fhealth                              → {\"status\":\"ok\"}\n\nGET    \u002Fapi\u002Fkiwi\u002Ftree?path=                 → directory tree (JSON)\nGET    \u002Fapi\u002Fkiwi\u002Ffile?path=                 → raw markdown + ETag\nPUT    \u002Fapi\u002Fkiwi\u002Ffile?path=                 → write + git commit + re-index\nDELETE \u002Fapi\u002Fkiwi\u002Ffile?path=                 → delete + git commit\nPOST   \u002Fapi\u002Fkiwi\u002Fbulk                       → multi-file write, one commit\nPOST   \u002Fapi\u002Fkiwi\u002Frename                     → atomic rename ({\"from\":\"...\",\"to\":\"...\"})\nPOST   \u002Fapi\u002Fkiwi\u002Ffile\u002Fappend?path=          → append content to file\n\nGET    \u002Fapi\u002Fkiwi\u002Fchanges?since=&limit=      → git-based change feed\nGET    \u002Fapi\u002Fkiwi\u002Fsearch?q=                  → full-text search (BM25)\nPOST   \u002Fapi\u002Fkiwi\u002Fsearch\u002Fsemantic            → vector search\n\nGET    \u002Fapi\u002Fkiwi\u002Fversions?path=             → git log for file\nGET    \u002Fapi\u002Fkiwi\u002Fversion?path=&version=     → content at commit\nGET    \u002Fapi\u002Fkiwi\u002Fdiff?path=&from=&to=       → unified diff\nGET    \u002Fapi\u002Fkiwi\u002Fblame?path=                → per-line attribution\n\nGET    \u002Fapi\u002Fkiwi\u002Fmeta?where=$.field=val     → structured query over frontmatter\nGET    \u002Fapi\u002Fkiwi\u002Fbacklinks?path=            → pages that link to this page\nGET    \u002Fapi\u002Fkiwi\u002Ftoc?path=                  → heading outline\nGET    \u002Fapi\u002Fkiwi\u002Fevents                     → SSE stream\nPOST   \u002Fapi\u002Fkiwi\u002Fresolve-links             → resolve [[wiki-links]] to permalinks\n\nGET    \u002Fapi\u002Fkiwi\u002Fstale                      → pages past their review date\nGET    \u002Fapi\u002Fkiwi\u002Fcontradictions             → pages with conflicting claims\nGET    \u002Fapi\u002Fkiwi\u002Fsearch\u002Fverified            → trust-ranked search (verified pages boosted)\nGET    \u002Fapi\u002Fkiwi\u002Fjanitor                    → knowledge health scan\nGET    \u002Fapi\u002Fkiwi\u002Fmemory\u002Freport              → episodic vs merged-from coverage (JSON)\n\nGET    \u002Fapi\u002Fkiwi\u002Fquery?q=                   → DQL query (TABLE, LIST, COUNT, DISTINCT)\nGET    \u002Fapi\u002Fkiwi\u002Fquery\u002Faggregate            → aggregation (count, avg, sum, min, max)\nPOST   \u002Fapi\u002Fkiwi\u002Fview\u002Frefresh              → refresh computed views\nPOST   \u002Fapi\u002Fkiwi\u002Fimport                    → bulk import from data source\nGET    \u002Fapi\u002Fkiwi\u002Fexport                    → export to JSONL\u002FCSV stream\nGET    \u002Fapi\u002Fkiwi\u002Fanalytics                 → knowledge health dashboard\nGET    \u002Fapi\u002Fkiwi\u002Fhealth-check?path=        → per-page health metrics\nGET    \u002Fapi\u002Fkiwi\u002Fcontext                   → schema + playbook + index in one call\n\nPOST   \u002Fapi\u002Fkiwi\u002Fshare                     → create a share link (password-protected)\nGET    \u002Fapi\u002Fkiwi\u002Fshare                     → list active share links\nDELETE \u002Fapi\u002Fkiwi\u002Fshare\u002F:id                 → revoke a share link\n\nPOST   \u002Fapi\u002Fkiwi\u002Fassets                     → upload binary asset (images, PDFs)\n```\n\nWrites accept `X-Actor` (git attribution), `X-Provenance` (lineage tracking), and `If-Match` (optimistic locking — 409 on conflict).\n\n---\n\n## Design principles\n\n1. **Files are the source of truth.** Every artifact is a plain markdown file. No proprietary format. Delete the search index — the files remain. `cat file.md` always works.\n\n2. **Two interfaces, one truth.** The web UI and the agent filesystem read\u002Fwrite the same files. No sync. No eventual consistency. One folder, two ways to access it.\n\n3. **Search is derivative.** The FTS5 index, the vector index, the metadata table — all rebuildable from files. `kiwifs reindex` and you're back. The folder is the truth, never the index.\n\n4. **Storage-agnostic.** KiwiFS depends on `open()`, `read()`, `write()`, `listdir()`. It doesn't care if the folder is on a laptop SSD, NFS, EFS, JuiceFS, or a FUSE-mounted S3 bucket.\n\n5. **Git as the WAL.** Instead of building a custom write-ahead log, every write is a git commit. Crash recovery, audit trail, tamper detection, replication — all for free.\n\n6. **Embeddable.** The Go library (`pkg\u002Fkiwi`) lets you embed KiwiFS in any Go application. The web UI components (`\u003CKiwiTree \u002F>`, `\u003CKiwiPage \u002F>`, `\u003CKiwiEditor \u002F>`, `\u003CKiwiSearch \u002F>`) are built for future standalone use as an npm package.\n\n---\n\n## POSIX compliance\n\nKiwiFS stores real files on a real filesystem — not blobs in a database. The degree of POSIX compliance depends on the access path:\n\n| Access path | POSIX level | Notes |\n|---|---|---|\n| **Direct filesystem** | Full | Real files, crash-safe atomic writes, mmap works |\n| **NFS mount** | Near-full | Userspace NFSv3, symlinks, open-unlink, advisory locking, stable handles across restarts |\n| **FUSE mount** | Near-full | Remote FUSE client, symlinks, directory rename, sub-second mtime, O_APPEND |\n| **WebDAV** | Partial | MOVE\u002FCOPY\u002FMKCOL\u002FDELETE, buffered writes with spill-to-disk |\n| **REST API** | N\u002FA | HTTP semantics (ETag concurrency, not POSIX) |\n| **S3 API** | N\u002FA | S3-compatible, not POSIX |\n| **MCP** | N\u002FA | Tool calls, not file ops |\n\n### What works\n\n| POSIX semantic | NFS | FUSE | How |\n|---|---|---|---|\n| **Atomic writes** | Yes | Yes | `write → fsync → rename(tmp, target) → fsync(dir)` — the gold-standard crash-safe pattern |\n| **rename(2)** | Yes | Yes | Files via pipeline (atomic); directories via bulk endpoint |\n| **O_APPEND** | Yes | Yes | FUSE fetches existing content on open, writes at correct EOF offset |\n| **O_TRUNC \u002F ftruncate** | Yes | Yes | NFS `Truncate()` with 64MB bounds; FUSE `Setattr` with `FATTR_SIZE` |\n| **Symlinks** | Yes | Yes | Real `os.Symlink` on NFS; `Content-Type: application\u002Fx-symlink` on FUSE + REST |\n| **readlink** | Yes | Yes | NFS via `os.Readlink`; FUSE via `\u002Fapi\u002Fkiwi\u002Freadlink`; REST API endpoint |\n| **Open-then-delete** | Yes | — | NFS defers deletion until last file handle closes (POSIX unlink semantics) |\n| **fsync** | Yes | Yes | NFS `Sync()` pushes through pipeline; FUSE `Fsync()` PUTs to server |\n| **Sub-second mtime** | — | Yes | FUSE `Getattr` reports `Mtimensec` from `Last-Modified` header |\n| **Advisory locking** | Yes | — | NFS has process-local `Lock()`\u002F`Unlock()` per file handle |\n| **Directory rename** | Yes | Yes | FUSE calls `\u002Fapi\u002Fkiwi\u002Frename-dir`; NFS uses `os.Rename` + bulk re-index |\n| **readdir** | Yes | Yes | Both hide internal dirs (`.git`, `.kiwi`) |\n| **stat** | Yes | Yes | Size, mode, mtime — real values, not synthetic |\n| **EFBIG on oversize** | Yes | Yes | 64MB `maxFileSize` limit returns proper errno \u002F HTTP error |\n| **mmap** | — | Passthrough | Works on NFS mount (kernel handles it); FUSE is over HTTP so no kernel mmap |\n| **Path safety** | Yes | Yes | `GuardPath` blocks traversal, null bytes, control chars, 255-byte segment limit |\n\n### Concurrency & durability\n\n- **Optimistic locking** — ETags (content hash). Writes with `If-Match` headers get HTTP 409 on conflict. `If-Match: *` is handled per RFC 7232 §3.1.\n- **Serialized writes** — the pipeline serializes all writes through a single mutex, so concurrent writers are safely queued regardless of protocol.\n- **Single-instance guard** — `flock(2)` on `.kiwi\u002Fserver.lock` prevents two servers from sharing the same data directory (SQLite + git corruption).\n- **Crash recovery** — stale `index.lock` files are cleaned by a background watcher (10s interval, 60s threshold). Git subprocesses receive SIGTERM before SIGKILL, giving them a chance to release locks.\n- **Line-ending integrity** — `core.autocrlf=false` + `* -text` in `.gitattributes` ensures ETags always match raw content. Writes to `.gitattributes` are blocked by the API.\n- **Frontmatter bomb protection** — YAML frontmatter blocks exceeding 64KB are silently treated as empty (headings still extracted).\n- **Stable NFS handles** — file handles are derived from `SHA-256(namespaceUUID + path)`, surviving server restarts. No more ESTALE.\n\n### What is intentionally not supported\n\n| POSIX semantic | Why |\n|---|---|\n| **Hard links** | Would break git versioning (one blob, multiple paths) |\n| **chmod \u002F chown** | No user\u002Fgroup model — auth is API-key \u002F OIDC, not POSIX uid\u002Fgid |\n| **POSIX ACLs** | Same reason — access control is at the HTTP\u002Fspace level |\n| **Extended attributes (xattr)** | Frontmatter serves the same purpose |\n| **Distributed locking** | Locks are process-local; use `If-Match` for cross-client concurrency |\n\n---\n\n## How it compares\n\n| | KiwiFS | agent-vfs | ChromaFS | Obsidian | Outline | Confluence |\n|---|---|---|---|---|---|---|\n| **Writable** (agents can create\u002Fupdate) | Yes | Yes | No (read-only) | Local only | API only | No |\n| **Agent-native** (`cat`\u002F`grep`\u002FNFS) | Yes | Virtual only | Read-only | Local only | API only | No |\n| **Web UI** (Notion-like) | Yes | No | No | Desktop | Yes | Yes |\n| **Versioned** (git audit trail) | Yes | No | No | No | Limited | Plugin ($$$) |\n| **Searchable** (FTS + vector) | Yes | No | Chroma | Plugins | Yes | Yes |\n| **Query language** (DQL) | Yes | No | No | Plugin | No | No |\n| **Data import** (18 sources) | Yes | No | No | No | API | No |\n| **Export** (JSONL\u002FCSV + embeddings) | Yes | No | No | No | Markdown | PDF only |\n| **Single binary** | Yes | No | No | No | No | No (SaaS) |\n| **Embeddable** (Go library) | Yes | No | No | No | No | No |\n| **Self-hosted** | Yes | Yes | Yes | N\u002FA | Yes | No |\n| **Files are the truth** | Yes | DB is truth | DB is truth | Yes | Postgres | Proprietary DB |\n\n---\n\n## Who this is for\n\n**AI agent builders** — Give your agent persistent, searchable memory it accesses with `cat`. Users browse the same files in a web UI with wiki links and graph view. Knowledge compounds across sessions instead of vanishing.\n\n**Teams replacing Confluence \u002F Notion** — Obsidian's file-first philosophy with a web UI anyone on your team can use. `git clone` your entire wiki. Self-hosted. No vendor lock-in.\n\n**Compliance-heavy industries** — Every change is a git commit with SHA-1 hash chain. Immutable audit trail. `git blame` for per-line attribution. An auditor can verify with standard tools.\n\n**DevOps \u002F platform teams** — Runbooks that agents update after every incident. Humans review in the UI. No more docs that rot after three months.\n\n---\n\n## Architecture\n\n```\n┌──────────────────────────────────────────────────────────┐\n│  KiwiFS                                    single Go binary\n│                                                          │\n│  ┌────────────────────────────────────────────────────┐  │\n│  │  Web UI (embedded via go:embed)                    │  │\n│  │  shadcn\u002Fui · BlockNote · react-markdown · Sigma.js │  │\n│  └────────────────────┬───────────────────────────────┘  │\n│                       │                                  │\n│  ┌────────────────────▼───────────────────────────────┐  │\n│  │  Access: REST :3333 · NFS :2049 · S3 :3334 · WebDAV│  │\n│  └────────────────────┬───────────────────────────────┘  │\n│                       │                                  │\n│  ┌────────────────────▼───────────────────────────────┐  │\n│  │  Core                                              │  │\n│  │  Storage · Git versioning · FTS5 + Vector search   │  │\n│  │  Watcher (fsnotify) · SSE events · Schema\u002Flint     │  │\n│  └────────────────────┬───────────────────────────────┘  │\n│                       │                                  │\n│  ┌────────────────────▼───────────────────────────────┐  │\n│  │  .git\u002F (audit WAL)  ·  .kiwi\u002Fstate\u002F (indexes)      │  │\n│  └────────────────────┬───────────────────────────────┘  │\n│                       │                                  │\n│  ┌────────────────────▼───────────────────────────────┐  │\n│  │  Filesystem: local · NFS · EFS · JuiceFS · FUSE-S3 │  │\n│  └────────────────────────────────────────────────────┘  │\n└──────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Inspired by\n\n- **[PocketBase](https:\u002F\u002Fpocketbase.io)** — single Go binary, zero config, just works. KiwiFS brings the same philosophy to knowledge infrastructure.\n- **[Obsidian](https:\u002F\u002Fobsidian.md)** — files are the database. Wiki links. Graph view. KiwiFS is Obsidian for the web — plus an agent interface and an API.\n- **[Karpathy's LLM Wiki](https:\u002F\u002Fgist.github.com\u002Fkarpathy\u002F442a6bf555914893e9891c11519de94f)** — raw sources in, compiled wiki out, agent maintains it. KiwiFS is the production runtime for this pattern.\n- **[Mintlify ChromaFS](https:\u002F\u002Fwww.mintlify.com\u002Fblog\u002Fhow-we-built-a-virtual-filesystem-for-our-assistant)** — the filesystem is the best agent interface. Agents already know `cat`\u002F`grep`\u002F`ls`.\n- **[Confluence](https:\u002F\u002Fwww.atlassian.com\u002Fsoftware\u002Fconfluence)** \u002F **[Notion](https:\u002F\u002Fnotion.so)** — great UIs, but your content is locked in their database. KiwiFS gives you the editing experience without the lock-in.\n\n## License\n\n[Business Source License 1.1](LICENSE) — free to use, self-host, and modify. The only restriction: you can't offer KiwiFS as a commercial hosted service. Each release converts to Apache 2.0 after 4 years.\n\nIf you want to offer KiwiFS as a managed service or need a commercial license, [get in touch](mailto:amelia.anh.lam@gmail.com).\n\n\"KiwiFS\" and the KiwiFS logo are trademarks of the KiwiFS Authors. See [LICENSE](LICENSE) for trademark usage guidelines.\n\n## Contributors\n\n\u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fkiwifs\u002Fkiwifs\u002Fgraphs\u002Fcontributors\">\n  \u003Cimg src=\"https:\u002F\u002Fcontrib.rocks\u002Fimage?repo=kiwifs\u002Fkiwifs&columns=6&max=20&v=2\" width=\"200\" \u002F>\n\u003C\u002Fa>\n","KiwiFS 是一个知识文件系统，允许智能代理写入、搜索、查询并信任存储的信息。其核心功能包括使用 `cat` 等命令进行文件写入操作，支持全文和语义搜索，以及通过结构化元数据进行复杂查询。KiwiFS 采用 Git 版本控制系统确保每一次写入都有不可变的审计追踪，同时提供了一个类似 Obsidian 和 Notion 的网页界面，方便人类用户阅读和浏览。该工具以 Go 语言编写，具有单个二进制文件、无需配置的特点，适合需要自托管的知识管理场景，如个人笔记库、团队协作文档平台或作为 AI 代理的数据存储解决方案。",2,"2026-06-11 02:40:47","CREATED_QUERY"]