[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-1465":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":10,"languages":10,"totalLinesOfCode":10,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":15,"forks30d":15,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":10,"archived":21,"fork":22,"defaultBranch":23,"hasWiki":22,"hasPages":22,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":25,"readmeContent":26,"aiSummary":27,"trendingCount":15,"starSnapshotCount":15,"syncStatus":28,"lastSyncTime":29,"discoverSource":30},1465,"LLM101n","karpathy\u002FLLM101n","karpathy","LLM101n: Let's build a Storyteller","",null,37294,2050,35,19,0,8,64,385,36,112.94,true,false,"master",[],"2026-06-12 04:00:09","# LLM101n: Let's build a Storyteller\n\n---\n\n**!!! NOTE: this course does not yet exist. It is current being developed by [Eureka Labs](https:\u002F\u002Feurekalabs.ai). Until it is ready I am archiving this repo !!!**\n\n---\n\n![LLM101n header image](llm101n.jpg)\n\n>  What I cannot create, I do not understand. -Richard Feynman\n\nIn this course we will build a Storyteller AI Large Language Model (LLM). Hand in hand, you'll be able to create, refine and illustrate little [stories](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Froneneldan\u002FTinyStories) with the AI. We are going to build everything end-to-end from basics to a functioning web app similar to ChatGPT, from scratch in Python, C and CUDA, and with minimal computer science prerequisites. By the end you should have a relatively deep understanding of AI, LLMs, and deep learning more generally.\n\n**Syllabus**\n\n- Chapter 01 **Bigram Language Model** (language modeling)\n- Chapter 02 **Micrograd** (machine learning, backpropagation)\n- Chapter 03 **N-gram model** (multi-layer perceptron, matmul, gelu)\n- Chapter 04 **Attention** (attention, softmax, positional encoder)\n- Chapter 05 **Transformer** (transformer, residual, layernorm, GPT-2)\n- Chapter 06 **Tokenization** (minBPE, byte pair encoding)\n- Chapter 07 **Optimization** (initialization, optimization, AdamW)\n- Chapter 08 **Need for Speed I: Device** (device, CPU, GPU, ...)\n- Chapter 09 **Need for Speed II: Precision** (mixed precision training, fp16, bf16, fp8, ...)\n- Chapter 10 **Need for Speed III: Distributed** (distributed optimization, DDP, ZeRO)\n- Chapter 11 **Datasets** (datasets, data loading, synthetic data generation)\n- Chapter 12 **Inference I: kv-cache** (kv-cache)\n- Chapter 13 **Inference II: Quantization** (quantization)\n- Chapter 14 **Finetuning I: SFT** (supervised finetuning SFT, PEFT, LoRA, chat)\n- Chapter 15 **Finetuning II: RL** (reinforcement learning, RLHF, PPO, DPO)\n- Chapter 16 **Deployment** (API, web app)\n- Chapter 17 **Multimodal** (VQVAE, diffusion transformer)\n\n**Appendix**\n\nFurther topics to work into the progression above:\n\n- Programming languages: Assembly, C, Python\n- Data types: Integer, Float, String (ASCII, Unicode, UTF-8)\n- Tensor: shapes, views, strides, contiguous, ...\n- Deep Learning frameworks: PyTorch, JAX\n- Neural Net Architecture: GPT (1,2,3,4), Llama (RoPE, RMSNorm, GQA), MoE, ...\n- Multimodal: Images, Audio, Video, VQVAE, VQGAN, diffusion\n","该项目旨在构建一个故事讲述者AI大型语言模型。通过从基础到高级的逐步教学，涵盖从大词模型到Transformer架构、注意力机制、优化算法等核心技术点，并最终实现一个类似ChatGPT的Web应用程序。整个过程使用Python、C和CUDA编程，适合希望深入了解AI、LLM及深度学习原理与实践的开发者或学生参与。此外，项目还涉及数据集处理、推理加速技术以及多模态内容生成等内容，为参与者提供了全面的学习体验。",2,"2026-06-11 02:43:57","top_all"]