[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-93336":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":13,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":15,"stars30d":15,"stars90d":14,"forks30d":14,"starsTrendScore":16,"compositeScore":17,"rankGlobal":9,"rankLanguage":9,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":19,"hasPages":19,"topics":21,"createdAt":9,"pushedAt":9,"updatedAt":22,"readmeContent":23,"aiSummary":24,"trendingCount":14,"starSnapshotCount":14,"syncStatus":16,"lastSyncTime":25,"discoverSource":26},93336,"inference-school","videlalvaro\u002Finference-school","videlalvaro","A hands-on Swift and Metal course for building LLM inference from first principles on Apple silicon, with 48 guided lessons, runnable exercises, a native macOS Studio, and a complete companion book.",null,"Swift",170,14,1,0,32,2,62.73,"Apache License 2.0",false,"main",[],"2026-07-22 04:02:08","# Inference School\n\nInference School is a problem-based course in which every exercise contributes to one\nsmall language-model inference engine for Apple Silicon. The problems are not\nstandalone interview puzzles. A dot product becomes GEMV, GEMV becomes Q\u002FK\u002FV\nprojections, those projections become attention, and the same path eventually\nloads a model and generates tokens.\n\nEach numerical operator has two required paths where a GPU mapping is part of\nthe lesson:\n\n1. A readable Swift CPU implementation that acts as the correctness oracle.\n2. A Metal implementation whose memory traffic, dispatch shape, and numerical\n   behavior you can explain and measure.\n\nThe finish line is not merely \"the tests pass.\" For each problem, you predict\nperformance, run an experiment, inspect the result, and integrate the operator\ninto the growing engine.\n\n## How you use it\n\nInference School Studio is the primary learning environment. It is a native macOS app\nwith a searchable lesson reader, editable Swift and Metal source files,\nout-of-process checks, persistent completion lists, diagrams, math rendering,\nand text scaling from 80% to 200%. The command line exposes the same lessons,\nlearner files, judges, and benchmarks.\n\n![Inference School Studio displaying the Start Here lesson and token-generation diagram](docs\u002Fassets\u002Finference-school-studio-start-here.png)\n\nFrom the repository root, package and open the Studio:\n\n```sh\nscripts\u002Fpackage-studio.sh debug\nopen \"dist\u002FInference School Studio.app\"\n```\n\nThe packaging script creates an ad hoc signed, sandboxed app for local use. It\ndoes not produce a Developer ID signature or a notarized distribution.\n\nIt opens at the reader-only orientation lesson `000`. Read\n[Start Here](docs\u002FSTART-HERE.md) for setup, local-execution boundaries, and the\nintended learning loop. Problems 001 through 047 are runnable, including the\nfinal profiling and systems module.\n\nThe packaged Studio is signed with App Sandbox. The first runnable lesson asks\nyou to choose a dedicated build folder through the macOS folder picker. Editable\nsources, compiler output, and generated learner executables stay under that\nfolder. The Studio host has the client entitlement required by WebKit on\nsupported macOS releases; its lesson, diagram, and editor assets are bundled,\nand built-in checks do not upload learner code. Command-line checks are\nseparate: they run with the permissions of your terminal session.\n\n## Companion book\n\nThe complete course is available as a 545-page PDF with native diagrams,\nexercises, and chapter-end solutions:\n\n[Download the Inference School Companion PDF](dist\u002FInference-School-Companion.pdf)\n\nRebuild and verify it with:\n\n```sh\nmake -C Book check\nopen \"dist\u002FInference-School-Companion.pdf\"\n```\n\nThe book toolchain requires Python 3, Pandoc, LuaLaTeX, latexmk, and\nGhostscript. Intermediate book files remain excluded from Git; the published\nPDF is versioned so readers can download it directly from the repository.\n\n## Project status\n\nThis repository contains the complete runnable 47-problem curriculum and its\neducational capstone inference engine:\n\n- Problem 001: Vector Dot Product\n- Problem 002: Tensor Storage and Strides\n- Problem 003: Transpose and Tiled Copy\n- Problem 004: Matrix-Vector Multiplication\n- Problem 005: Matrix-Matrix Multiplication\n- Problem 006: Build a Roofline Measurement\n- Problem 007: ReLU, GELU, and SiLU\n- Problem 008: SwiGLU Feed-Forward Gate\n- Problem 009: Numerically Stable Softmax\n- Problem 010: RMSNorm\n- Problem 011: Residual Streams and Precision\n- Problem 012: Fuse Norm, Scale, and Projection Input\n- Problem 013: Embedding Lookup and Tied Output Weights\n- Problem 014: Q\u002FK\u002FV Projections and Head Views\n- Problem 015: Rotary Position Embeddings\n- Problem 016: Causal Attention for One Head\n- Problem 017: Multi-Head Attention\n- Problem 018: MHA, MQA, and GQA\n- Problem 019: Online Softmax Attention\n- Problem 020: Tiled Fused Attention\n- Problem 021: Sliding-Window Attention\n- Problem 022: Preallocate and Append K\u002FV\n- Problem 023: Cached Single-Token Attention\n- Problem 024: KV Layout Shootout\n- Problem 025: Shared KV Heads\n- Problem 026: Ring-Buffer Sliding Cache\n- Problem 027: Paged KV Allocation\n- Problem 028: Quantized KV Cache\n- Problem 029: Symmetric INT8 Quantization\n- Problem 030: Per-Channel and Groupwise Scales\n- Problem 031: Pack and Unpack INT4\n- Problem 032: Dequantize Then GEMV\n- Problem 033: Fused Q4 GEMV\n- Problem 034: Quantization Error Propagation\n- Problem 035: One Decoder Transformer Block\n- Problem 036: Parse a Model Weight Format\n- Problem 037: Tokenization and Detokenization\n- Problem 038: Logits and Sampling\n- Problem 039: Prompt Prefill\n- Problem 040: Autoregressive Decode\n- Problem 041: Buffer Reuse and Memory Planning\n- Problem 042: Checkpoint Parity and First Divergence\n- Problem 043: Fuse RMSNorm and Q\u002FK\u002FV Projections\n- Problem 044: Profile Prefill and Decode Separately\n- Problem 045: Static and Continuous Batching\n- Problem 046: Speculative Decoding\n- Problem 047: Capstone Inference Engine\n- CPU and Metal starter implementations\n- Separate canonical CPU and Metal solutions\n- A shared correctness judge\n- Vector-dot, roofline, KV-layout, fused-Q4-GEMV, buffer-plan, fused-QKV,\n  prefill\u002Fdecode profile, and capstone report commands\n\nThe capstone runs the complete educational model on the CPU reference backend\nand executes fused QKV plus RoPE as an explicitly labeled Metal verification\nslice. It does not claim a pretrained model or complete Metal generation.\n\n## Requirements\n\n- An Apple Silicon Mac\n- macOS 15 or newer\n- Xcode or the Xcode command-line tools with Swift and Metal\n\nVerify the local tools:\n\n```sh\nswift --version\nxcrun --find metal\n```\n\n## Command reference\n\n```sh\nswift run inference-school list\nswift run inference-school learn 001\nswift run inference-school show 004\nswift run inference-school check 004 --cpu\n```\n\nLearner checks are expected to fail value cases before you implement an\nexercise. For problems with a Metal stage, implement both the Swift exercise\nand its matching starter kernel, then run:\n\n```sh\nswift run inference-school check 004 --metal\nswift run inference-school check 004\n```\n\nThe canonical implementations let you verify the harness itself:\n\n```sh\nswift run inference-school check 004 --solution\nswift test\n```\n\nUse a release build for measurements:\n\n```sh\nswift run -c release inference-school benchmark 001\nswift run -c release inference-school benchmark 006\nswift run -c release inference-school benchmark 024\nswift run -c release inference-school benchmark 033\nswift run inference-school benchmark 041 --tokens 128 --cached-tokens 128\nswift run -c release inference-school benchmark 043 --tokens 32 --iterations 20\nswift run -c release inference-school profile 044 --prompt-tokens 16 --trials 7\nswift run -c release inference-school capstone --prompt \"ab c.\" --max-tokens 4\n```\n\n## Course map\n\nStart with [Problem 000](Problems\u002F000-start-here\u002FREADME.md), which defines the\ncourse workflow and core model vocabulary. Then use\n[Anatomy of One Token](docs\u002FONE-TOKEN.md) as a compact map of the complete\ninference path before the course takes it apart. Keep the\n[Math Primer](docs\u002FMATH-PRIMER.md) nearby and read only the section needed for\nthe current problem.\n\nThe complete sequence is in [The Curriculum](docs\u002FCURRICULUM.md). It moves\nthrough:\n\n1. Tensor storage, reductions, GEMV, GEMM, and performance measurement\n2. Activations, SwiGLU, softmax, normalization, and fusion\n3. Embeddings, Q\u002FK\u002FV, RoPE, and several forms of attention\n4. KV-cache layouts, paging, windows, sharing, and quantization\n5. Weight quantization and fused low-bit matrix-vector kernels\n6. Model loading, tokenization, prefill, decode, sampling, and memory planning\n7. Profiling, batching, speculative decoding, and a measured capstone engine\n\n## Repository layout\n\n```text\nProblems\u002F                    Full tutorials and required experiments\nSources\u002FInferenceSchoolCore\u002F         Judges and reusable host-side infrastructure\nSources\u002FInferenceSchoolExercises\u002F    Files the learner edits\nSources\u002FInferenceSchoolSolutions\u002F    Canonical answers, kept out of the exercise module\nSources\u002FInferenceSchoolCLI\u002F          list, show, check, and benchmark commands\nSources\u002FInferenceSchoolStudio\u002F       Native macOS lesson reader and workbench\nTests\u002F                       Tests for judges, infrastructure, and answers\nWeb\u002F                         Source and lockfiles for bundled browser resources\nBook\u002F                        Companion-book generator and publication checks\nPackaging\u002F                   App metadata and sandbox entitlements\ndocs\u002F                        Course design, roadmap, math, and system context\n```\n\nThe teaching and completion rules are documented in\n[docs\u002FCOURSE-DESIGN.md](docs\u002FCOURSE-DESIGN.md).\n\n## Security\n\nInference School executes learner code. The packaged Studio confines that work to an\nApp Sandbox and a user-selected folder, but command-line checks inherit the\npermissions of the terminal that launches them. Read [Start Here](docs\u002FSTART-HERE.md)\nbefore running checks, and report vulnerabilities according to\n[SECURITY.md](SECURITY.md).\n\n## License\n\nInference School is licensed under the [Apache License 2.0](LICENSE). Dependency\nattributions are recorded in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)\nand included in packaged Studio builds.\n","这是一个面向 Apple Silicon 平台的 LLM 推理引擎实践课程，通过 48 个渐进式实验，从零构建基于 Swift 和 Metal 的轻量级语言模型推理系统。核心特点是双路径实现（可验证的 Swift CPU 实现 + 可分析的 Metal GPU 实现），覆盖 GEMV、Q\u002FK\u002FV 投影、注意力机制到端到端 token 生成全流程，并配套原生 macOS 学习应用与 545 页图文详尽的配套教材。适合希望深入理解 LLM 推理底层原理、掌握 Metal 异构计算优化、并在 macOS 环境下开展模型部署实践的开发者与学习者。","2026-07-16 02:30:09","CREATED_QUERY"]