[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-2020":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":16,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":17,"rankGlobal":9,"rankLanguage":9,"license":9,"archived":18,"fork":18,"defaultBranch":19,"hasWiki":20,"hasPages":18,"topics":21,"createdAt":9,"pushedAt":9,"updatedAt":22,"readmeContent":23,"aiSummary":24,"trendingCount":15,"starSnapshotCount":15,"syncStatus":25,"lastSyncTime":26,"discoverSource":27},2020,"llm-wiki","kothari-nikunj\u002Fllm-wiki","kothari-nikunj","Personal Wiki",null,"TypeScript",137,20,136,4,0,1,3.97,false,"main",true,[],"2026-06-12 02:00:35","# LLM Wiki\n\nYour personal knowledge base, compiled by AI.\n\n## What This Is\n\nA system for building a personal wiki where an LLM reads your data -- writing, tweets, messages, bookmarks -- and compiles it into interconnected articles. Think Wikipedia, but the subject is one life and mind.\n\nThree layers:\n\n1. **Raw sources** (`data\u002F`) -- Your writing, tweets, messages, bookmarks. You drop files here.\n2. **Compiled wiki** (`wiki\u002F`) -- Markdown articles generated by Claude Code. Organized by theme, linked together with `[[wikilinks]]`. You never edit these directly.\n3. **Web viewer** (`pages\u002Fwiki\u002F`) -- A Next.js app that renders the wiki as a browsable website. Password-protected. Deploy locally or to any host.\n\nThe brain is a Claude Code skill (`.claude\u002Fskills\u002Fwiki\u002FSKILL.md`). It tells Claude how to read your raw data, understand what it means, and write articles that capture that understanding. All wiki content lives as plain markdown files. You can use any AI, any editor, any tool -- file over app.\n\n---\n\n## Prerequisites\n\n- **Node.js 18+** (check with `node --version`)\n- **Python 3** (check with `python3 --version`)\n- **Claude Code** (the CLI for Claude -- [install instructions](https:\u002F\u002Fdocs.anthropic.com\u002Fen\u002Fdocs\u002Fclaude-code))\n- **macOS** (required only for iMessage and WhatsApp ingestion)\n\n---\n\n## Quick Start (5 minutes)\n\n### 1. Clone and install\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FYOUR_USERNAME\u002Fllm-wiki.git\ncd llm-wiki\nnpm install\n```\n\n### 2. Configure environment\n\n```bash\ncp .env.example .env.local\n```\n\nEdit `.env.local` and set:\n\n```\nWIKI_PASSWORD=choose_a_password_for_the_wiki\nJWT_SECRET=any_random_string_at_least_32_characters_long\n```\n\nThe `WIKI_PASSWORD` is what you'll type to access the wiki in a browser. The `JWT_SECRET` signs the authentication token.\n\n### 3. Add your data\n\nDrop your source files into the `data\u002F` directory. See the [Data Sources](#data-sources) section below for instructions on each format.\n\nAt minimum, drop a few `.md` files into `data\u002Fwriting\u002F` to get started.\n\n### 4. Ingest your data\n\n```bash\npython3 ingest.py\n```\n\nThis converts your raw data into individual markdown entries in `raw\u002Fentries\u002F`. Each entry is one tweet, one blog post, one day of conversation. The ingest step is mechanical -- no LLM needed.\n\n### 5. Compile the wiki\n\nOpen Claude Code in this directory and run:\n\n```\n\u002Fwiki absorb all\n```\n\nClaude reads every entry in `raw\u002Fentries\u002F`, understands what it means, and writes wiki articles in `wiki\u002F`. This is where the magic happens. For a few hundred entries, this takes 5-15 minutes.\n\n### 6. Browse your wiki\n\n```bash\nnpm run dev\n```\n\nOpen [http:\u002F\u002Flocalhost:3000\u002Fwiki](http:\u002F\u002Flocalhost:3000\u002Fwiki). Enter the password you set in step 2.\n\nYou'll see a sidebar with categories and articles, a search bar, and your compiled wiki. Every `[[wikilink]]` in the articles is clickable. The table of contents appears in the right margin for longer articles.\n\n---\n\n## Data Sources\n\n### Substack \u002F Blog Posts\n\nIf you have a Substack or other blog with markdown exports:\n\n```bash\n# Clone the scraper\ngit clone https:\u002F\u002Fgithub.com\u002Ftimf34\u002FSubstack2Markdown \u002Ftmp\u002Fsubstack-scraper\n\n# Set up Python environment\npython3 -m venv \u002Ftmp\u002Fsubstack-env\nsource \u002Ftmp\u002Fsubstack-env\u002Fbin\u002Factivate\npip install -r \u002Ftmp\u002Fsubstack-scraper\u002Frequirements.txt\n\n# Scrape your Substack\npython \u002Ftmp\u002Fsubstack-scraper\u002Fsubstack_scraper.py \\\n  --url https:\u002F\u002FYOUR-SUBSTACK.substack.com \\\n  --directory data\u002Fwriting \\\n  --images\n\n# Deactivate the venv when done\ndeactivate\n```\n\nYour posts are now in `data\u002Fwriting\u002F` as individual `.md` files.\n\nIf you already have markdown files from another blog platform, just copy them into `data\u002Fwriting\u002F`.\n\n### Twitter\u002FX Archive\n\nDownload your full tweet history:\n\n1. Go to [x.com](https:\u002F\u002Fx.com) -> Settings -> Your Account -> Download an archive of your data\n2. Wait for the email (can take 24-48 hours)\n3. Download and unzip the archive\n4. Copy the tweets file:\n\n```bash\ncp ~\u002FDownloads\u002Ftwitter-archive\u002Fdata\u002Ftweets.js data\u002Ftweets\u002F\n```\n\nThe ingest script parses the Twitter archive format automatically. It extracts original tweets (skipping replies and retweets), along with dates, media URLs, and mentions.\n\n### X Bookmarks\n\nIf you use [ft-cli](https:\u002F\u002Fgithub.com\u002Fafar1\u002Ffieldtheory-cli) to sync your X bookmarks:\n\n1. Your bookmarks are at `~\u002F.ft-bookmarks\u002Fbookmarks.jsonl`\n2. The ingest script automatically finds and processes them -- no manual copying needed\n\nEach bookmark becomes an entry with author, engagement stats, and the full tweet text.\n\n### iMessages (macOS only)\n\nExtract conversations from your Mac's iMessage database:\n\n1. **Grant Full Disk Access** to your terminal app:\n   - Open System Settings -> Privacy & Security -> Full Disk Access\n   - Add your terminal (Terminal.app, iTerm2, Warp, etc.)\n   - Restart your terminal\n\n2. **Configure the script** (optional):\n   Open `ingest_imessage.py` and adjust:\n   - `YOUR_NAME` -- Change `\"Me\"` to your actual name\n   - `TOP_N` -- Number of top contacts to process (default: 100)\n   - `TS_START` -- How far back to go (default: 2016)\n   - `MIN_MSG_LEN` -- Minimum message length (default: 15 chars)\n\n3. **Run the ingest:**\n\n```bash\npython3 ingest_imessage.py\n```\n\nThe script reads `~\u002FLibrary\u002FMessages\u002Fchat.db` directly and resolves phone numbers\u002Femails to contact names using your Address Book. Messages are grouped by contact and day -- each day's conversation becomes one entry.\n\n### WhatsApp (macOS only)\n\nExtract conversations from the WhatsApp desktop app:\n\n1. **Install WhatsApp desktop** (not WhatsApp Web) and let it fully sync\n2. **Configure the script** (optional):\n   Open `ingest_whatsapp.py` and adjust `YOUR_NAME`, `TOP_N`, `MIN_MSG_LEN`\n3. **Run:**\n\n```bash\npython3 ingest_whatsapp.py\n```\n\nThe script reads the local WhatsApp SQLite database at `~\u002FLibrary\u002FGroup Containers\u002Fgroup.net.whatsapp.WhatsApp.shared\u002FChatStorage.sqlite`.\n\n### Other Data Formats\n\n**Markdown or plain text files:**\nDrop any `.md` or `.txt` files into `data\u002Fwriting\u002F`. The ingest script picks them up automatically.\n\n**EPUB files:**\nDrop `.epub` files into `data\u002Fwriting\u002F`. The wiki skill handles extraction -- when you run `\u002Fwiki ingest`, Claude will write a custom parser for any format it doesn't recognize.\n\n**Apple Notes:**\nExport your notes as `.txt` or `.html` files and drop them in `data\u002Fwriting\u002F`.\n\n**Anything else:**\nThe wiki skill is designed to handle unknown formats. Drop your data in `data\u002F`, run `\u002Fwiki ingest`, and Claude will figure out the structure and write a parser.\n\n---\n\n## Commands\n\nAll commands are run inside Claude Code. Type them in the Claude Code prompt.\n\n### `\u002Fwiki ingest`\n\nConverts source data in `data\u002F` into individual `.md` entries in `raw\u002Fentries\u002F`. Runs the Python ingest scripts. This step is mechanical -- it just restructures data into a standard format.\n\nIf `ingest.py` doesn't cover a data format you have, Claude will write a custom parser.\n\n### `\u002Fwiki absorb [date-range]`\n\nThe core compilation step. Claude reads entries from `raw\u002Fentries\u002F` and writes wiki articles in `wiki\u002F`.\n\nDate range options:\n- `\u002Fwiki absorb all` -- Process every entry\n- `\u002Fwiki absorb last 30 days` -- Process recent entries\n- `\u002Fwiki absorb 2024` -- Process entries from 2024\n- `\u002Fwiki absorb 2024-03` -- Process entries from March 2024\n- `\u002Fwiki absorb 2024-03-22` -- Process entries from a specific date\n\nDefault (no argument): absorb entries from the last 30 days.\n\nWhat happens during absorb:\n1. Claude reads each entry chronologically\n2. For each entry, it checks the wiki index for matching articles\n3. It either updates existing articles or creates new ones\n4. Every 15 entries, it checkpoints: rebuilds the index, audits quality, checks for cramming (putting too much into one article) or thinning (too many stubs)\n\nThe absorb process creates articles organized by theme, not chronology. A page about a person isn't a timeline -- it's about who they are and what they mean.\n\n### `\u002Fwiki query \u003Cquestion>`\n\nAsk questions about your wiki. Claude navigates the compiled articles, follows links, and synthesizes an answer. It saves the response to `outputs\u002F` for future reference.\n\nExamples:\n- `\u002Fwiki query What are my core beliefs about building products?`\n- `\u002Fwiki query Who are the people I've worked with most closely?`\n- `\u002Fwiki query What patterns repeat across my career transitions?`\n\nQuery is read-only -- it never modifies wiki articles.\n\n### `\u002Fwiki cleanup`\n\nAudits and enriches every article. Claude reads all articles, checks structure and quality, rewrites articles that read like chronological dumps, identifies missing articles, fixes broken wikilinks, and rebuilds the index.\n\nRun this after a large absorb to improve quality.\n\n### `\u002Fwiki breakdown`\n\nFinds and creates missing articles. Claude scans existing articles for named entities (people, places, companies, concepts) that are mentioned but don't have their own pages. It ranks candidates by reference count and creates new articles for the most-referenced ones.\n\n### `\u002Fwiki status`\n\nShows stats: total entries absorbed, articles by category, most-connected articles, orphans (articles with no links to\u002Ffrom), and pending entries.\n\n### `\u002Fwiki rebuild-index`\n\nRegenerates `wiki\u002F_index.md` and `wiki\u002F_backlinks.json` from the current state of wiki articles. Run this if the index gets out of sync.\n\n### `\u002Fwiki reorganize`\n\nSteps back and rethinks the wiki structure. Claude reads the index, samples articles, and asks: should anything be merged? Split? Should new categories exist? Are there orphaned articles? Missing patterns?\n\n---\n\n## Architecture\n\n### Directory Structure\n\n```\nllm-wiki\u002F\n  data\u002F                          # Your raw source files (not committed to git)\n    writing\u002F                     # Blog posts, essays, markdown files\n    tweets\u002F                      # Twitter\u002FX archive (tweets.js)\n  raw\u002F\n    entries\u002F                     # One .md per entry (generated by ingest)\n  wiki\u002F                          # The compiled knowledge base\n    _index.md                    # Master index with aliases\n    _backlinks.json              # Reverse link index\n    _absorb_log.json             # Tracks which entries have been absorbed\n    {category}\u002F                  # Articles organized by category\n      article-name.md            # Individual wiki articles\n  outputs\u002F                       # Query results and reports\n  .claude\u002Fskills\u002Fwiki\u002FSKILL.md   # The wiki skill (the brain)\n  pages\u002Fwiki\u002F                    # Next.js pages for the web viewer\n  components\u002Fwiki\u002F               # React components for rendering\n  utils\u002Fwiki-loader.ts           # Markdown-to-HTML pipeline\n  ingest.py                      # Multi-format data ingest\n  ingest_imessage.py             # iMessage extractor (macOS)\n  ingest_whatsapp.py             # WhatsApp extractor (macOS)\n```\n\n### How Articles Work\n\nEvery wiki article is a markdown file with YAML frontmatter:\n\n```yaml\n---\ntitle: Article Title\ntype: person | concept | company | era | pattern | ...\ncreated: 2024-01-15\nlast_updated: 2024-03-22\nrelated: [\"[[other\u002Farticle]]\", \"[[another\u002Farticle]]\"]\nsources: [\"entry-id-1\", \"entry-id-2\"]\n---\n\n# Article Title\n\nContent organized by theme, not chronology.\n\n## Section Heading\n\nText with [[wikilinks]] to other articles.\n```\n\nArticles link to each other with `[[wikilinks]]`. The web viewer renders these as clickable links. The `_backlinks.json` file tracks which articles link to which, enabling \"linked from\" navigation.\n\n### Categories\n\nCategories (directories inside `wiki\u002F`) emerge from your data. The wiki skill doesn't pre-create them. Common ones that tend to appear:\n\n| Category | What goes here |\n|----------|---------------|\n| `people\u002F` | Named individuals |\n| `companies\u002F` | Companies and organizations |\n| `concepts\u002F` | Ideas and mental models |\n| `philosophies\u002F` | Articulated positions and beliefs |\n| `patterns\u002F` | Recurring behavioral cycles |\n| `eras\u002F` | Major life phases |\n| `interests\u002F` | Hobbies, passions, topics of attention |\n| `strategies\u002F` | Named strategies and approaches |\n| `projects\u002F` | Things built with serious commitment |\n| `places\u002F` | Cities, neighborhoods, buildings |\n| `decisions\u002F` | Inflection points with enumerated reasoning |\n\n### Web Viewer Components\n\nThe web viewer is a standard Next.js app with four main components:\n\n- **`wiki-layout.tsx`** -- Sidebar with category navigation, search, and random article button\n- **`wiki-index.tsx`** -- Home page content showing article count and welcome text\n- **`wiki-article.tsx`** -- Article renderer with header, metadata, and styled body\n- **`wiki-toc.tsx`** -- Right margin column with table of contents, related articles, and backlinks\n\nThe `utils\u002Fwiki-loader.ts` module reads markdown files from `wiki\u002F`, parses frontmatter with `gray-matter`, converts markdown to HTML with `unified`\u002F`remark`\u002F`rehype`, and transforms `[[wikilinks]]` into clickable HTML links.\n\n### Authentication\n\nThe wiki is password-protected. The middleware (`middleware.ts`) checks for a JWT cookie on every `\u002Fwiki\u002F*` request. If missing or invalid, it redirects to `\u002Fwiki\u002Flogin`.\n\nThe login page posts the password to `\u002Fapi\u002Fwiki\u002Fauth`, which checks it against `WIKI_PASSWORD` and sets a `wiki_auth` cookie valid for 30 days.\n\n---\n\n## Customizing\n\n### Change the sidebar title\n\nEdit `components\u002Fwiki\u002Fwiki-layout.tsx`. Find `\u003CSidebarTitle>Wiki\u003C\u002FSidebarTitle>` and change the text.\n\n### Change the home page text\n\nEdit `components\u002Fwiki\u002Fwiki-index.tsx`. Replace the welcome text with whatever you want.\n\n### Add custom fonts\n\nEdit `global.css` to add `@font-face` declarations and update the `html` font-family.\n\n### Change the color scheme\n\nThe wiki uses these colors:\n- `#0080ff` -- Links and accents\n- `#1a1a1a` -- Primary text\n- `#37352f` -- Article body text\n- `#6b6b6b` -- Secondary text\n- `#9b9a97` -- Metadata text\n- `#c4c4c0` -- Muted text\n- `#e8e8e6` -- Borders\n- `#f5f5f4` -- Badges and code backgrounds\n\nAll colors are defined inline in the styled-components. Search for hex codes to change them globally.\n\n### Disable authentication\n\nIf you only run the wiki locally and don't need a password, delete `middleware.ts` and the login page. The wiki will be accessible without authentication.\n\n### Add a new data source\n\n1. Write an ingest script (Python or Node) that reads your data format and writes `.md` files to `raw\u002Fentries\u002F`\n2. Each file should have YAML frontmatter with: `id`, `date`, `time`, `source_type`, `title`, `tags`\n3. Run the script, then run `\u002Fwiki absorb all`\n\n---\n\n## Deploying\n\n### Railway (recommended)\n\n1. Push your repo to GitHub\n2. Create a new project on [Railway](https:\u002F\u002Frailway.com) and connect your repo\n3. Add environment variables in the Railway dashboard:\n   - `WIKI_PASSWORD` -- your chosen password\n   - `JWT_SECRET` -- your secret string\n\n4. The wiki directory needs to be committed for the build to work. Update `.gitignore` to track your wiki content:\n\n```bash\n# Remove the wiki exclusion from .gitignore, or commit wiki files explicitly\ngit add wiki\u002F\ngit commit -m \"Add wiki content\"\ngit push\n```\n\nRailway will build and deploy automatically. Add a domain in the Railway dashboard to make your wiki accessible.\n\n### Other hosts\n\nThe project is a standard Next.js app. It works with any host that supports Node.js:\n\n```bash\nnpm run build\nnpm start\n```\n\nSet `WIKI_PASSWORD` and `JWT_SECRET` as environment variables on your host.\n\n---\n\n## Source Hierarchy\n\nNot all data sources are equal. The absorb step weights them differently:\n\n| Priority | Source | Signal | How it's used |\n|----------|--------|--------|---------------|\n| 1 | **Writing** (blog posts, essays) | Your clearest, most developed thinking | Nearly every post seeds or enriches an article. Forms the wiki backbone. |\n| 2 | **Tweets** (original, not replies\u002FRTs) | Well-formed but short public thoughts | Clusters by theme. Extends ideas from writing. Standalone articles only for ideas not written up elsewhere. |\n| 3 | **Bookmarks** | Interest signals, not your own thinking | Reveals what topics pull your attention. Updates existing articles with \"what you follow.\" Never creates per-bookmark articles. |\n| 4 | **Messages** (iMessage, WhatsApp) | Raw, unfiltered, highest noise | Highly selective. Only patterns that repeat or moments that mattered. A lunch plan is noise. A 2am career conversation is signal. |\n\n## Absorb Strategy\n\nFor large datasets, don't run `\u002Fwiki absorb all` on everything at once. Layer sources in order:\n\n```\n# Step 1: Writing first (establishes the backbone)\n# Drop writing in data\u002Fwriting\u002F, run ingest, then:\n\u002Fwiki absorb all\n\n# Step 2: Add tweets (extends and reinforces)\n# Copy tweets.js to data\u002Ftweets\u002F, run ingest, then:\n\u002Fwiki absorb all\n\n# Step 3: Add bookmarks (maps interests)\n# Ensure ~\u002F.ft-bookmarks\u002Fbookmarks.jsonl exists, run ingest, then:\n\u002Fwiki absorb all\n\n# Step 4: Add messages (enriches relationships)\n# Run ingest_imessage.py and\u002For ingest_whatsapp.py, then:\n\u002Fwiki absorb all\n```\n\nAfter each layer, run `\u002Fwiki cleanup` to audit quality before adding the next source. This produces much better results than dumping everything in at once.\n\nFor very large batches (5000+ entries per source), the absorb will take time. Use parallel agents if available — split by time period (e.g., tweets 2010-2017, 2018-2023, 2024+) and process simultaneously.\n\n## Workflow\n\nA typical ongoing workflow:\n\n1. **Collect data.** Over days\u002Fweeks, accumulate writing, tweets, bookmarks, conversations.\n2. **Ingest.** Run `python3 ingest.py` (and message scripts if applicable) to convert raw data into entries.\n3. **Absorb.** Run `\u002Fwiki absorb last 30 days` in Claude Code. Claude reads new entries and updates\u002Fcreates articles.\n4. **Browse.** Run `npm run dev` and explore your wiki at localhost:3000\u002Fwiki.\n5. **Refine.** Run `\u002Fwiki cleanup` to improve article quality. Run `\u002Fwiki breakdown` to find missing articles.\n6. **Query.** Use `\u002Fwiki query` to ask questions across your entire knowledge base.\n7. **Repeat.** The wiki compounds over time. Each absorb cycle makes it richer.\n\n### User Corrections\n\nThe LLM will sometimes get facts wrong — especially relationship origins, timelines, and who worked where. Review articles and correct errors by telling Claude directly:\n\n```\n\"I didn't meet Alex at Company X. We met at a party in 2013.\"\n\"Sam and Jordan both worked at the first startup together, then the second.\"\n\"It's Michael, not Mike.\"\n```\n\nClaude will fix the articles across the wiki. This is normal and expected — the human reviews, the LLM maintains.\n\n---\n\n## Deploying\n\n### Local only (default)\n\nThe `.gitignore` excludes wiki content by default. This means your wiki lives only on your machine. Run `npm run dev` to browse it locally.\n\n### Deploy to the web\n\nIf you want your wiki accessible online (still password-protected):\n\n1. Remove the wiki exclusion from `.gitignore`:\n\n```bash\n# Edit .gitignore and remove these lines:\n# wiki\u002F**\u002F*.md\n# wiki\u002F_*.json\n```\n\n2. Commit and push:\n\n```bash\ngit add wiki\u002F\ngit commit -m \"Add wiki content\"\ngit push\n```\n\n3. Deploy to [Railway](https:\u002F\u002Frailway.com) (or any Next.js host). Set `WIKI_PASSWORD` and `JWT_SECRET` as environment variables in the hosting dashboard.\n\n---\n\n## Troubleshooting\n\n### \"WIKI_PASSWORD not configured\"\n\nYou forgot to create `.env.local` or didn't set the `WIKI_PASSWORD` variable. Run:\n\n```bash\ncp .env.example .env.local\n# Edit .env.local and set both variables\n```\n\n### Build fails with module not found\n\nRun `npm install` again. If a specific package is missing, install it:\n\n```bash\nnpm install \u003Cpackage-name>\n```\n\n### iMessage ingest says \"database not found\"\n\nYour terminal doesn't have Full Disk Access. Go to System Settings -> Privacy & Security -> Full Disk Access and add your terminal app. Restart the terminal after granting access.\n\n### WhatsApp ingest says \"database not found\"\n\nMake sure you have the WhatsApp desktop app installed (not just WhatsApp Web) and that it has fully synced your messages. The database file is at `~\u002FLibrary\u002FGroup Containers\u002Fgroup.net.whatsapp.WhatsApp.shared\u002FChatStorage.sqlite`.\n\n### Wiki shows no articles\n\nMake sure you've run both ingest and absorb:\n\n```bash\npython3 ingest.py          # Creates raw\u002Fentries\u002F\n# Then in Claude Code:\n\u002Fwiki absorb all           # Creates wiki\u002F articles\n```\n\nThe web viewer reads from `wiki\u002F`, not from `raw\u002Fentries\u002F`.\n\n### Styled-components flash of unstyled content\n\nThis is a known issue with styled-components and SSR. The `compiler.styledComponents: true` setting in `next.config.js` handles this for production builds. In development, you may see a brief flash on first load.\n\n### How do I add more data later?\n\n1. Drop new files into `data\u002F`\n2. Run `python3 ingest.py` again (it's idempotent -- won't duplicate existing entries)\n3. Run `\u002Fwiki absorb last 30 days` (or whatever date range covers your new data)\n\n### How do I start over?\n\nDelete the generated content and re-run:\n\n```bash\nrm -rf raw\u002Fentries\u002F*.md wiki\u002F**\u002F*.md wiki\u002F_*.json outputs\u002F*.md\npython3 ingest.py\n# Then in Claude Code:\n\u002Fwiki absorb all\n```\n\n### Date serialization error during build\n\nIf you see `object (\"[object Date]\") cannot be serialized as JSON`, this means `gray-matter` parsed a YAML date as a JavaScript `Date` object. The `wiki-loader.ts` handles this with `toDateString()`, but if you add new frontmatter fields with dates, wrap them in quotes in the YAML: `created: \"2024-01-15\"`.\n\n### iMessage: most messages show as empty\n\nOn newer macOS versions, most iMessage text is stored in the `attributedBody` column (an NSAttributedString blob), not the `text` column. The `ingest_imessage.py` script extracts text from both. If you're getting very few messages, check that the `extract_text_from_blob` function is working — it uses a regex pattern to find readable text within the binary blob.\n\n### WhatsApp: very few old messages\n\nThe WhatsApp desktop app often only syncs recent messages (last 1-2 years). Pre-2024 messages may have very little text data available. This is a WhatsApp limitation, not a bug in the ingest script.\n\n### Tweet count seems low\n\nThe ingest script filters out replies and retweets by default — only original tweets are ingested. This typically reduces 17,000+ tweets down to ~5,000-6,000 original posts. This is intentional: replies are conversational noise, and retweets are other people's content.\n\n### Contact names showing as phone numbers\n\nThe iMessage script resolves names using the macOS Address Book database. If contacts show as phone numbers, either:\n- The contact isn't in your Address Book\n- The phone number format doesn't match (the script normalizes to last 10 digits)\n- The Address Book database isn't accessible (check Full Disk Access permissions)\n\n---\n\n## Technical Notes\n\n### How the wikilink pipeline works\n\n1. Raw markdown with `[[wikilinks]]` is read from wiki files\n2. `transformWikilinks()` in `wiki-loader.ts` converts `[[slug]]` and `[[slug|Label]]` to `\u003Ca href=\"\u002Fwiki\u002Fslug\" class=\"wikilink\">Label\u003C\u002Fa>` HTML\n3. The transformed markdown is processed through `remark-parse` → `remark-gfm` → `remark-rehype` → `rehype-slug` → `rehype-stringify`\n4. The resulting HTML is passed to the React component via `dangerouslySetInnerHTML`\n5. Global CSS rules in `.wiki-page` suppress the site's default link pseudo-elements to prevent style conflicts\n\n### How auth works\n\n1. `middleware.ts` intercepts all `\u002Fwiki\u002F*` requests (except `\u002Fwiki\u002Flogin`)\n2. It checks for a `wiki_auth` cookie containing a JWT\n3. If missing or invalid, it redirects to `\u002Fwiki\u002Flogin`\n4. The login page posts the password to `\u002Fapi\u002Fwiki\u002Fauth`\n5. If correct, the API signs a JWT with `jose` and sets an httpOnly cookie (30-day expiry)\n\n### How the ingest scripts find your data\n\n| Source | Location | How it's found |\n|--------|----------|---------------|\n| Writing | `data\u002Fwriting\u002F` | Recursive glob for `*.md` files |\n| Tweets | `data\u002Ftweets\u002Ftweets.js` | Explicit path |\n| Bookmarks | `~\u002F.ft-bookmarks\u002Fbookmarks.jsonl` | Home directory path |\n| iMessages | `~\u002FLibrary\u002FMessages\u002Fchat.db` | macOS system path |\n| WhatsApp | `~\u002FLibrary\u002FGroup Containers\u002Fgroup.net.whatsapp.WhatsApp.shared\u002FChatStorage.sqlite` | macOS app container |\n| Contacts | `~\u002FLibrary\u002FApplication Support\u002FAddressBook\u002FSources\u002F*\u002FAddressBook-v22.abcddb` | macOS Address Book |\n\n---\n\n## Credits\n\nInspired by:\n\n- [Andrej Karpathy's LLM Knowledge Bases gist](https:\u002F\u002Fgist.github.com\u002Fkarpathy\u002F442a6bf555914893e9891c11519de94f) — The original concept of LLM-compiled personal wikis\n- [Farza's Farzapedia](https:\u002F\u002Fgist.github.com\u002Ffarzaa\u002Fc35ac0cfbeb957788650e36aabea836d) — The Claude Code skill for wiki compilation, and the Wikipedia-style viewer concept\n- [Steph Ango's File Over App](https:\u002F\u002Fstephango.com\u002Ffile-over-app) — The philosophy of storing data in universal file formats\n- [Substack2Markdown](https:\u002F\u002Fgithub.com\u002Ftimf34\u002FSubstack2Markdown) — Blog post scraping\n- [ft-cli](https:\u002F\u002Fgithub.com\u002Fnichochar\u002Fft-cli) — X\u002FTwitter bookmark syncing\n\n---\n\n## License\n\nMIT\n","LLM Wiki 是一个基于AI的个人知识库系统，用于将用户的写作、推文、消息和书签等原始数据编译成相互关联的文章。该项目的核心功能包括通过Claude Code技能读取用户提供的原始数据，并自动生成Markdown格式的文章；这些文章按主题组织并通过`[[wikilink]]`链接起来。最终成果可通过一个由Next.js构建的Web应用程序浏览，支持本地部署或任何主机部署。此项目适用于希望以结构化方式整理个人知识与经历的用户，特别是那些拥有大量非结构化文本资料并希望通过AI技术进行智能整理的人士。",2,"2026-06-11 02:47:36","CREATED_QUERY"]