[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-78036":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":8,"htmlUrl":8,"language":9,"languages":8,"totalLinesOfCode":8,"stars":10,"forks":11,"watchers":12,"openIssues":13,"contributorsCount":13,"subscribersCount":13,"size":13,"stars1d":13,"stars7d":14,"stars30d":15,"stars90d":13,"forks30d":13,"starsTrendScore":13,"compositeScore":16,"rankGlobal":8,"rankLanguage":8,"license":8,"archived":17,"fork":17,"defaultBranch":18,"hasWiki":19,"hasPages":19,"topics":20,"createdAt":8,"pushedAt":8,"updatedAt":21,"readmeContent":22,"aiSummary":23,"trendingCount":13,"starSnapshotCount":13,"syncStatus":24,"lastSyncTime":25,"discoverSource":26},78036,"how-llms-work","ynarwal\u002Fhow-llms-work","ynarwal",null,"HTML",144,22,104,0,1,40,4.09,false,"main",true,[],"2026-06-12 02:03:45","# How LLMs Actually Work\n\nA visual, interactive guide to how large language models are built — from raw internet text to a conversational assistant.\n\n**Live site:** https:\u002F\u002Fynarwal.github.io\u002Fhow-llms-work\u002F\n\nBased on Andrej Karpathy's [Intro to Large Language Models](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=zjkBMFhNj_g) lecture.\n\n---\n\n## What's inside\n\n- **Data Collection** — how the web is scraped and filtered into training data (Common Crawl, FineWeb)\n- **Tokenization** — how text is broken into subword tokens via Byte Pair Encoding (BPE)\n- **Neural Network Training** — the loss function, gradient descent, and what a forward pass looks like\n- **Inference & Sampling** — how the model generates text token by token, and how temperature works\n- **The Base Model** — what a model knows after pre-training and what it can't do yet\n- **Post-Training** — RLHF, instruction tuning, and how a base model becomes an assistant\n- **LLM Psychology** — hallucinations, context windows, and how to think about what models \"know\"\n- **RAG** — retrieval-augmented generation: embeddings, vector search, and context injection\n- **Full Pipeline Summary** — end-to-end visual of every stage\n\n---\n\n## Files\n\n| File | Description |\n|------|-------------|\n| `index.html` | Main site (v2 redesign) |\n| `v1.html` | Original dark-theme version |\n| `transcript.txt` | Full Karpathy lecture transcript |\n| `council.py` | LLM council fact-checker (runs via `uv run council.py`) |\n| `report.html` | Latest council fact-check report |\n\n---\n\n## HN discussion\n\n[Posted to Hacker News](https:\u002F\u002Fnews.ycombinator.com\u002Fitem?id=47886517) and generated heated debate, mostly about it being LLM-generated. Fair point — but the content isn't the AI's. Every claim, figure, and framing is traced directly to Karpathy's lecture, not hallucinated by a model.\n\n## Vibe check\n\nThe code and content in this repo is mostly LLM-generated (Claude via Claude Code). The ideas, direction, and editorial decisions are mine — the implementation was largely written by AI. The council fact-checker exists precisely because of this: automated content warrants automated verification.\n","该项目是一个可视化、交互式的指南，旨在解释大型语言模型从原始网络文本到对话助手的构建过程。核心功能包括数据收集、分词、神经网络训练、推理与采样、基础模型介绍、后训练技术如RLHF和指令调优等，并深入探讨了LLM的心理学特性及检索增强生成技术。项目基于Andrej Karpathy关于大型语言模型的讲座内容，适用于对AI特别是自然语言处理领域感兴趣的开发者、研究人员以及任何希望了解LLM工作原理的学习者。此外，尽管大部分代码和内容由AI生成，但所有信息均直接来源于Karpathy的演讲，确保了准确性。",2,"2026-06-11 03:56:23","CREATED_QUERY"]