[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-74095":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":14,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":10,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":21,"hasPages":23,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":25,"readmeContent":26,"aiSummary":27,"trendingCount":16,"starSnapshotCount":16,"syncStatus":28,"lastSyncTime":29,"discoverSource":30},74095,"guppylm","arman-bd\u002Fguppylm","arman-bd","A ~9M parameter LLM that talks like a small fish.","",null,"Python",3208,282,8,1,0,4,46,12,29.36,false,"main",true,[],"2026-06-12 02:03:22","\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fguppy.png\" alt=\"GuppyLM\" width=\"400\"\u002F>\n\u003C\u002Fp>\n\n\u003Ch1 align=\"center\">GuppyLM\u003C\u002Fh1>\n\u003Cp align=\"center\">\u003Cem>A ~9M parameter LLM that talks like a small fish.\u003C\u002Fem>\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Farman-bd\u002Fguppylm-60k-generic\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🤗_Dataset-guppylm--60k-blue\" alt=\"Dataset\"\u002F>\u003C\u002Fa> \n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Farman-bd\u002Fguppylm-9M\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🤗_Model-guppylm--9M-orange\" alt=\"Model\"\u002F>\u003C\u002Fa> \n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Farman-bd\u002Fguppylm\u002Fblob\u002Fmain\u002FLICENSE\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-green\" alt=\"License\"\u002F>\u003C\u002Fa>\n  \u003Cbr\u002F>\n  \u003Ca href=\"https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Farman-bd\u002Fguppylm\u002Fblob\u002Fmain\u002Ftrain_guppylm.ipynb\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTrain_in-Colab-F9AB00?logo=googlecolab\" alt=\"Train\"\u002F>\u003C\u002Fa> \n  \u003Ca href=\"https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Farman-bd\u002Fguppylm\u002Fblob\u002Fmain\u002Fuse_guppylm.ipynb\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FChat_in-Colab-F9AB00?logo=googlecolab\" alt=\"Chat\"\u002F>\u003C\u002Fa>\n  \u003Cbr\u002F>\n  \u003Ca href=\"https:\u002F\u002Fwww.linkedin.com\u002Fpulse\u002Fbuild-your-own-language-model-5-minutes-i-made-mine-hossain--supif\u002F\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FArticle-LinkedIn-0A66C2?logo=linkedin\" alt=\"LinkedIn Article\"\u002F>\u003C\u002Fa> \n  \u003Ca href=\"https:\u002F\u002Farman-bd.medium.com\u002Fbuild-your-own-llm-in-5-minutes-i-made-mine-talk-like-a-fish-e20c338a3d14\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FArticle-Medium-000000?logo=medium\" alt=\"Medium Article\"\u002F>\u003C\u002Fa>\n  \u003Cbr\u002F>\u003Cbr\u002F>\n  \u003Ca href=\"https:\u002F\u002Farman-bd.github.io\u002Fguppylm\u002F\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTry_in-Browser-64ffda?style=for-the-badge&logo=webassembly&logoColor=white\" alt=\"Browser Demo\"\u002F>\u003C\u002Fa>\n\u003C\u002Fp>\n\n---\n\n> **This project exists to show that training your own language model is not magic.**\n> No PhD required. No massive GPU cluster. One Colab notebook, 5 minutes, and you have a working LLM that you built from scratch — data generation, tokenizer, model architecture, training loop, and inference. If you can run a notebook, you can train a language model.\n>\n> It won't produce a billion-parameter model that writes essays. But it will show you exactly how every piece works — from raw text to trained weights to generated output — so the big models stop feeling like black boxes.\n\n---\n\n```\nYou> hi guppy\nGuppy> hi there. i just found a nice spot near the rock. the temperature feels nice.\n\nYou> are you hungry\nGuppy> yes. always yes. i will swim to the top right now. i promise to eat all of it.\n\nYou> do you like bubbles\nGuppy> i love bubbles. they make the water feel slightly different.\n\nYou> what is the meaning of life\nGuppy> food. the answer is always food.\n\nYou> tell me a joke\nGuppy> what did the fish say when it hit the wall. dam.\n\nYou> do you love me\nGuppy> you're my favorite big shape. my mouth are happy when you're here.\n\nYou> goodnight guppy\nGuppy> ok sleep time. i was following a bubble but now i'll stop. goodnight tank. goodnight water.\n```\n\n---\n\n## What is GuppyLM?\n\nGuppyLM is a tiny language model that pretends to be a fish named Guppy. It speaks in short, lowercase sentences about water, food, light, and tank life. It doesn't understand human abstractions like money, phones, or politics — and it's not trying to.\n\nIt's trained from scratch on 60K synthetic conversations across 60 topics, runs on a single GPU in ~5 minutes, and produces a model small enough to run in a browser.\n\n---\n\n## Architecture\n\n| | |\n|---|---|\n| **Parameters** | 8.7M |\n| **Layers** | 6 |\n| **Hidden dim** | 384 |\n| **Heads** | 6 |\n| **FFN** | 768 (ReLU) |\n| **Vocab** | 4,096 (BPE) |\n| **Max sequence** | 128 tokens |\n| **Norm** | LayerNorm |\n| **Position** | Learned embeddings |\n| **LM head** | Weight-tied with embeddings |\n\nVanilla transformer. No GQA, no RoPE, no SwiGLU, no early exit. As simple as it gets.\n\n---\n\n## Personality\n\nGuppy:\n- Speaks in short, lowercase sentences\n- Experiences the world through water, temperature, light, vibrations, and food\n- Doesn't understand human abstractions\n- Is friendly, curious, and a little dumb\n- Thinks about food a lot\n\n**60 topics:** greetings, feelings, temperature, food, light, water, tank, noise, night, loneliness, bubbles, glass, reflection, breathing, swimming, colors, taste, plants, filter, algae, snails, scared, excited, bored, curious, happy, tired, outside, cats, rain, seasons, music, visitors, children, meaning of life, time, memory, dreams, size, future, past, name, weather, sleep, friends, jokes, fear, love, age, intelligence, health, singing, TV, and more.\n\n---\n\n## Quick Start\n\n### Try in Browser (no install needed)\n\n[![Try in Browser](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTry_in-Browser-64ffda?logo=webassembly)](https:\u002F\u002Farman-bd.github.io\u002Fguppylm\u002F)\n\nRuns entirely in your browser via WebAssembly. Downloads a quantized ONNX model (~10 MB) and runs inference locally — no server, no API keys.\n\n### Chat with Guppy in Colab\n\n[![Open in Colab](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FChat_in-Colab-F9AB00?logo=googlecolab)](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Farman-bd\u002Fguppylm\u002Fblob\u002Fmain\u002Fuse_guppylm.ipynb)\n\nDownloads the pre-trained model from HuggingFace and lets you chat. Just run all cells.\n\n### Train your own\n\n[![Open in Colab](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTrain_in-Colab-F9AB00?logo=googlecolab)](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Farman-bd\u002Fguppylm\u002Fblob\u002Fmain\u002Ftrain_guppylm.ipynb)\n\n1. Set runtime to **T4 GPU**\n2. **Run all cells** — downloads dataset, trains tokenizer, trains model, tests it\n3. Upload to HuggingFace or download locally\n\n### Chat locally\n\n```bash\npip install torch tokenizers\npython -m guppylm chat\n```\n\n```\nYou> the cat is looking at you\nGuppy> i hide behind the plant when the furry one comes.\n\nYou> it is raining outside\nGuppy> i think rain is the best thing about outside.\n```\n\nIn interactive chat mode, the conversation grows and quickly runs into the 128-token limit, reducing quality.\nYou can also invoke chat with a single prompt, and exit after the response:\n\n```bash\npython -m guppylm chat --prompt \"tell me a joke\"\n```\n\n\n---\n\n## Dataset\n\n**[arman-bd\u002Fguppylm-60k-generic](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Farman-bd\u002Fguppylm-60k-generic)** on HuggingFace.\n\n| | |\n|---|---|\n| Samples | 60,000 (57K train \u002F 3K test) |\n| Format | `{\"input\": \"...\", \"output\": \"...\", \"category\": \"...\"}` |\n| Categories | 60 |\n| Generation | Synthetic template composition |\n\n```python\nfrom datasets import load_dataset\nds = load_dataset(\"arman-bd\u002Fguppylm-60k-generic\")\nprint(ds[\"train\"][0])\n# {'input': 'hi guppy', 'output': 'hello. the water is nice today.', 'category': 'greeting'}\n```\n\n---\n\n## Project Structure\n\n```\nguppylm\u002F\n├── config.py               Hyperparameters (model + training)\n├── model.py                Vanilla transformer\n├── dataset.py              Data loading + batching\n├── train.py                Training loop (cosine LR, AMP)\n├── generate_data.py        Conversation data generator (60 topics)\n├── eval_cases.py           Held-out test cases\n├── prepare_data.py         Data prep + tokenizer training\n└── inference.py            Chat interface\n\ntools\u002F\n├── make_colab.py           Generates Colab notebooks\n├── export_onnx.py          Export model to ONNX (quantized uint8)\n├── export_dataset.py       Push dataset to HuggingFace\n└── dataset_card.md         HuggingFace dataset README\n\ndocs\u002F\n├── index.html              Browser demo (ONNX + WASM)\n├── download.sh             Download model.onnx + tokenizer from HF\n├── model.onnx              Quantized uint8 (~10 MB)\n├── tokenizer.json          BPE tokenizer\n└── guppy.png               Logo (transparent)\n```\n\n---\n\n## Design Decisions\n\n**Why no system prompt?** Every training sample had the same one. A 9M model can't conditionally follow instructions — the personality is baked into the weights. Removing it saves ~60 tokens per inference.\n\n**Why single-turn only?** Multi-turn degraded at turn 3-4 due to the 128-token context window. A fish that forgets is on-brand, but garbled output isn't. Single-turn is reliable.\n\n**Why vanilla transformer?** GQA, SwiGLU, RoPE, and early exit add complexity that doesn't help at 9M params. Standard attention + ReLU FFN + LayerNorm produces the same quality with simpler code.\n\n**Why synthetic data?** A fish character with consistent personality needs consistent training data. Template composition with randomized components (30 tank objects, 17 food types, 25 activities) generates ~16K unique outputs from ~60 templates.\n\n---\n\n## License\n\nMIT\n\n\u003C!-- GitAds-Verify: UW3VONXNDRGEZ2O7RF4N2PFHSUR3DUJB -->\n","GuppyLM 是一个约900万参数的语言模型，其对话风格模仿小鱼。该项目使用Python开发，通过简单的Colab笔记本即可完成从数据生成、分词器创建到模型训练和推理的全过程，旨在展示语言模型训练并非高不可攀的技术。GuppyLM的核心特点在于其极简性和教育意义，适合对自然语言处理感兴趣但缺乏相关背景知识的学习者快速入门，以及希望以低成本方式探索语言模型工作原理的研究人员。尽管它无法与大型商用模型相提并论，但对于理解语言模型内部机制提供了一个直观且易于上手的平台。",2,"2026-06-11 03:48:47","high_star"]