[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92874":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":14,"stars30d":15,"stars90d":14,"forks30d":14,"starsTrendScore":14,"compositeScore":16,"rankGlobal":9,"rankLanguage":9,"license":17,"archived":18,"fork":18,"defaultBranch":19,"hasWiki":18,"hasPages":18,"topics":20,"createdAt":9,"pushedAt":9,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":14,"starSnapshotCount":14,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},92874,"texts-to-transformer","Doriandarko\u002Ftexts-to-transformer","Doriandarko","Train a tiny Transformer from scratch on your iMessage history, entirely on your Mac.",null,"Python",390,37,94,0,107,54.74,"MIT License",false,"main",[21,22,23,24,25,26],"apple-silicon","imessage","local-first","machine-learning","mlx","transformer","2026-07-22 04:02:07","# Texts to Transformer\n\nTrain a tiny language model from scratch on your iMessage history, entirely on your Mac.\n\nThis repository contains the complete pipeline: safe Messages database snapshotting, text\nextraction and pseudonymization, leakage-resistant dataset splits, tokenizer training, a custom\ndecoder-only Transformer, MLX training, evaluation, memorization checks, model export, and a local\nterminal chat interface.\n\nNothing is pretrained. The tokenizer and model both start from zero.\n\n> [!IMPORTANT]\n> This builds a small personal style model, not a generally capable assistant. It can learn your\n> phrasing, rhythm, slang, and common responses, but it will not reliably reason or answer factual\n> questions. The resulting model may memorize private text and must remain private.\n\n## What you will build\n\nThe default small preset is a 4-layer, 1.38M-parameter decoder-only Transformer with a custom\n4,096-token byte-level BPE tokenizer and a 256-token context window. A larger 6.16M-parameter\npreset is included for unusually large message histories.\n\n```mermaid\nflowchart LR\n    A[\"Messages chat.db\"] -->|\"read-only SQLite backup\"| B[\"Private snapshot\"]\n    B --> C[\"Extract + pseudonymize\"]\n    C --> D[\"Conversation sessions\"]\n    D --> E[\"Chronological train \u002F validation \u002F test\"]\n    E --> F[\"Train custom tokenizer\"]\n    F --> G[\"Initialize Transformer randomly\"]\n    G --> H[\"Train + evaluate with MLX\"]\n    H --> I[\"Local reply generator\"]\n```\n\nAn example development run used roughly 8M training tokens and produced a 1.38M-parameter model\nthat generated short replies in the owner's writing style.\n\n## Safety and privacy\n\nRead [the privacy documentation](docs\u002Fprivacy.md) before running the data pipeline.\n\n- `~\u002FLibrary\u002FMessages\u002Fchat.db` is opened in SQLite read-only mode and is never modified.\n- Processing happens from a consistent private backup under `work\u002F`, never from the live database.\n- Attachments are never opened or copied.\n- Handles and chat identifiers are replaced with keyed HMAC pseudonyms before JSONL is written.\n- URLs, email addresses, and phone-number-shaped strings are redacted by default.\n- Raw messages are never printed in normal logs.\n- Datasets, tokenizers, checkpoints, and final weights are excluded from Git.\n- No command uploads data or sends an iMessage. `chat` only prints a suggestion in the terminal.\n\nPseudonymization is not anonymization. Keep `work\u002F` and `outputs\u002F` on a FileVault-protected Mac and\nnever commit, upload, or share them.\n\n## Requirements\n\n- An Apple Silicon Mac (M1 or newer)\n- macOS 14 or newer\n- At least 16 GB of unified memory recommended\n- Enough free disk space for a private copy of `chat.db` and training artifacts\n- [Homebrew](https:\u002F\u002Fbrew.sh\u002F) or another way to install `uv`\n- Full Disk Access for Terminal, Codex, or whichever app runs the snapshot command\n\nThe project uses Python 3.11 and pins MLX 0.32.0. It does not require PyTorch.\n\n## Install\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FDoriandarko\u002Ftexts-to-transformer.git\ncd texts-to-transformer\n\n# Skip this if uv is already installed.\nbrew install uv\n\nuv sync\nuv run imessage-mlx doctor\n```\n\n`doctor` verifies Apple Silicon, MLX Metal support, disk space, Git ignore coverage, private\ndirectory permissions, and read-only access to the Messages database.\n\n### Grant Full Disk Access\n\nIf `safe_to_snapshot_real_data` is `false`, open:\n\n```text\nSystem Settings → Privacy & Security → Full Disk Access\n```\n\nEnable the application running the command, completely restart that application, and rerun:\n\n```bash\nuv run imessage-mlx doctor\n```\n\nDo not copy the live database manually or change its permissions as a workaround.\n\n## Train your model\n\nRun these commands from the repository root. The commands print aggregate counts and metrics, not\nmessage text.\n\n### 1. Create a safe database snapshot\n\n```bash\nuv run imessage-mlx snapshot --config configs\u002Fdata.yaml\n```\n\nThis uses SQLite's online backup API, writes `work\u002Fsnapshot\u002Fchat.db`, hashes the snapshot, and runs\n`PRAGMA quick_check`.\n\n### 2. Inspect the local Messages schema\n\n```bash\nuv run imessage-mlx inspect-schema \\\n  --database work\u002Fsnapshot\u002Fchat.db \\\n  --output work\u002Fschema\u002Fschema.json\n```\n\nApple changes the Messages schema between macOS versions, so the extractor inspects the local\nschema instead of assuming an internet example is correct.\n\n### 3. Build the private dataset\n\n```bash\nuv run imessage-mlx prepare --config configs\u002Fdata.yaml\nuv run imessage-mlx privacy-audit\n```\n\nThis stage:\n\n1. Recovers ordinary text and Apple typedstream `attributedBody` text.\n2. Filters reactions, system events, deleted messages, and attachment-only rows.\n3. Redacts obvious identifiers and pseudonymizes database identities.\n4. Groups messages into six-hour conversation sessions.\n5. Removes duplicate sessions.\n6. Creates chronological 90\u002F5\u002F5 splits with seven-day guard bands.\n7. Verifies that no duplicate session hash appears across splits.\n\nThe command stops instead of silently continuing when recovery or privacy checks fail.\n\n### 4. Train the tokenizer\n\n```bash\nuv run imessage-mlx train-tokenizer \\\n  --train work\u002Fsplits\u002Ftrain.jsonl \\\n  --output outputs\u002Ftokenizer \\\n  --vocab-size 4096\n```\n\nOnly the training split is used. The byte-level BPE tokenizer preserves emoji, casing, punctuation,\nmultilingual text, slang, and unusual spelling.\n\n### 5. Encode the corpus and select a model size\n\n```bash\nuv run imessage-mlx corpus-stats \\\n  --splits work\u002Fsplits \\\n  --tokenizer outputs\u002Ftokenizer \\\n  --output work\u002Ftokens\n```\n\nRead `work\u002Freports\u002Fmodel-selection.json`. The project refuses real training below one million\ntokens and selects the largest preset supported by the corpus. A low token-to-parameter ratio is\nclearly marked as a memorization-prone experiment.\n\n### 6. Train from random initialization\n\nMost people should use the selected 1M preset:\n\n```bash\nuv run imessage-mlx train \\\n  --config configs\u002Fmodel-1m.yaml \\\n  --data work\u002Ftokens \\\n  --tokenizer outputs\u002Ftokenizer \\\n  --output outputs\u002Fruns\u002Fmy-model\n```\n\nIf `model-selection.json` explicitly selects `model-7m`, use `configs\u002Fmodel-7m.yaml` instead.\n\nTraining uses next-token cross-entropy, AdamW, learning-rate warmup and cosine decay, gradient\nclipping, compiled MLX updates, validation-based checkpoint selection, and resumable checkpoints.\n\nResume an interrupted run with:\n\n```bash\nuv run imessage-mlx train \\\n  --config configs\u002Fmodel-1m.yaml \\\n  --data work\u002Ftokens \\\n  --tokenizer outputs\u002Ftokenizer \\\n  --output outputs\u002Fruns\u002Fmy-model \\\n  --resume-from outputs\u002Fruns\u002Fmy-model\u002Flast\n```\n\n### 7. Evaluate the untouched test split\n\n```bash\nuv run imessage-mlx evaluate \\\n  --checkpoint outputs\u002Fruns\u002Fmy-model\u002Fbest \\\n  --data work\u002Ftokens \\\n  --output outputs\u002Fevaluation.json\n```\n\nThe report includes overall and `me`-turn perplexity, a unigram baseline, exact train n-gram overlap\naggregates, and obvious-PII pattern counts. Matching private text is never persisted in the report.\n\n### 8. Export the local model\n\n```bash\nuv run imessage-mlx export \\\n  --checkpoint outputs\u002Fruns\u002Fmy-model\u002Fbest \\\n  --metrics outputs\u002Fevaluation.json \\\n  --output outputs\u002Ffinal\n```\n\nThe exported directory includes inference weights, model configuration, tokenizer, aggregate\nmetrics, and dataset hashes. Optimizer state and source messages are excluded.\n\n### 9. Generate reply suggestions\n\n```bash\nuv run imessage-mlx chat --model outputs\u002Ffinal\n```\n\nExample interaction:\n\n```text\nLocal MLX reply generator. Type \u002Fquit to exit. Nothing will be sent.\nother: hey, are you around later?\nme: yeah should be! what time?\n```\n\nThe prompt labeled `other:` is the incoming message. The text labeled `me:` is the model's suggested\nreply. The command remembers earlier turns until `\u002Fquit`, but it never sends anything.\n\nFor shorter, less chaotic generations:\n\n```bash\nuv run imessage-mlx chat \\\n  --model outputs\u002Ffinal \\\n  --temperature 0.5 \\\n  --top-p 0.8 \\\n  --max-new-tokens 24 \\\n  --repetition-penalty 1.2\n```\n\n## What this model can and cannot do\n\nIt can learn:\n\n- Common greetings and sign-offs\n- Your average response length\n- Punctuation, emoji, slang, and casing patterns\n- Frequently repeated conversational structures\n\nIt does not reliably learn:\n\n- Arithmetic or factual knowledge\n- Multi-step reasoning\n- Long-term memory beyond its context window\n- The capabilities of a pretrained assistant model\n\nMaking the architecture larger without adding more unique training text usually increases\nmemorization rather than intelligence. If you want general reasoning plus personal style, fine-tune\na pretrained model instead; this repository intentionally demonstrates true from-scratch training.\n\n## Included model presets\n\n| Preset | Layers | Width | Heads | MLP width | Context | Vocabulary | Approx. parameters |\n| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| `model-1m` | 4 | 128 | 4 | 384 | 256 | 4,096 | 1.38M |\n| `model-7m` | 6 | 256 | 8 | 768 | 512 | 4,096 | 6.16M |\n\nSee [the architecture guide](docs\u002Farchitecture.md) for the model, data, and checkpoint design.\n\n## Validate everything\n\n```bash\nuv run ruff check .\nuv run ruff format --check .\nuv run pytest -q\ngit diff --check\ngit ls-files work outputs\n```\n\nThe final command must print nothing. The test suite uses synthetic SQLite and JSONL fixtures only.\nIt covers snapshot immutability, attributed-body decoding, extraction accounting, pseudonymization,\nsplit isolation, tokenizer behavior, causal masking, one-batch overfitting, checkpoint resume,\ncompiled MLX training, evaluation, export, and fresh-process inference.\n\nAfter a real run, perform the aggregate Gate A-D audit:\n\n```bash\nuv run imessage-mlx audit \\\n  --run outputs\u002Fruns\u002Fmy-model \\\n  --model outputs\u002Ffinal\n```\n\nThe audit writes aggregate evidence to `work\u002Freports\u002Fcompletion-audit.json` and exits nonzero unless\nevery required gate passes.\n\n## Troubleshooting\n\nSee [the troubleshooting guide](docs\u002Ftroubleshooting.md) for Full Disk Access failures, insufficient\ndata, interrupted runs, strange output, and privacy audit failures.\n\n## Project structure\n\n```text\nconfigs\u002F                    data and model presets\nsrc\u002Fimessage_mlx\u002Fdata\u002F      snapshot, decoding, extraction, privacy, sessions, splits\nsrc\u002Fimessage_mlx\u002Fmodel\u002F     decoder-only Transformer implementation\nsrc\u002Fimessage_mlx\u002Ftokenizer\u002F local tokenizer training\nsrc\u002Fimessage_mlx\u002Ftrain.py   MLX training loop\nsrc\u002Fimessage_mlx\u002Fevaluate.py held-out and memorization evaluation\nsrc\u002Fimessage_mlx\u002Fgenerate.py local autoregressive generation\ntests\u002F                      synthetic-only test suite\nwork\u002F                       ignored private datasets and reports\noutputs\u002F                    ignored private tokenizers, checkpoints, and models\n```\n\n## License\n\n[MIT](LICENSE)\n","这是一个在 Apple Silicon Mac 上本地训练微型 Transformer 语言模型的开源项目，专为个性化风格建模设计。它从零开始训练 tokenizer 和模型，完整支持 iMessage 历史数据的安全快照、文本提取、HMAC 伪匿名化、泄漏鲁棒的数据集划分、MLX 框架下的轻量级 decoder-only Transformer 训练及本地终端对话交互。模型参数仅约 138 万，上下文窗口 256 token，不依赖预训练，专注复现用户个人表达习惯（如用词、节奏、惯用语），但不具备通用推理或事实问答能力。适用于希望在完全离线、隐私优先环境下构建个人写作风格模型的技术用户。",2,"2026-07-11 02:30:09","CREATED_QUERY"]