[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-83312":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":9,"totalLinesOfCode":9,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":9,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":9,"rankLanguage":9,"license":9,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":21,"hasPages":21,"topics":9,"createdAt":9,"pushedAt":9,"updatedAt":23,"readmeContent":24,"aiSummary":25,"trendingCount":16,"starSnapshotCount":16,"syncStatus":26,"lastSyncTime":27,"discoverSource":28},83312,"LiteRT","google-ai-edge\u002FLiteRT","google-ai-edge","LiteRT, successor to TensorFlow Lite. is Google's On-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization",null,"https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT","C++",2539,339,25,104,0,30,32,90,29.59,false,"main","2026-06-12 02:04:33","# LiteRT\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\".\u002Fg3doc\u002Fsources\u002Flitert_logo.png\" alt=\"LiteRT Logo\"\u002F>\n\u003C\u002Fp>\n\nGoogle's on-device runtime for high-performance ML & GenAI deployment on edge platforms.\n\n📖 [Get Started](#-installation) | 🤝 [Contributing](#-contributing) | 📜 [License](#-license) | 🛡 [Security Policy](SECURITY.md) | 📄 [Documentation](https:\u002F\u002Fai.google.dev\u002Fedge\u002Flitert)\n\n---\n\n## 🛠 Build Status\n\n| Nightly Builds | Continuous Builds | Other Builds |\n| :--- | :--- | :--- |\n| [![Linux Nightly Wheel](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Flinux_nightly_wheel.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Flinux_nightly_wheel.yml)\u003Cbr>[![macOS Nightly Wheel](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fmacos_nightly_wheel.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fmacos_nightly_wheel.yml)\u003Cbr>[![Windows Nightly Wheel](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fwindows_nightly_wheel.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fwindows_nightly_wheel.yml) | [![macOS arm64](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fmacos-arm64.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fmacos-arm64.yml)\u003Cbr>[![Linux x86_64](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Flinux_x86_64.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Flinux_x86_64.yml)\u003Cbr>[![Windows x86_64](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fwindows_x86_64.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fwindows_x86_64.yml) | [![CMake Android Linux x86_64](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fcmake_android_linux_x86_64.yml\u002Fbadge.svg)](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Factions\u002Fworkflows\u002Fcmake_android_linux_x86_64.yml) |\n\n---\n\n## 📖 LiteRT\n\nLiteRT continues the legacy of TensorFlow Lite as the trusted, high-performance runtime for on-device AI. Featuring advanced GPU\u002FNPU acceleration, LiteRT delivers superior ML & GenAI performance, making on-device ML inference easier than ever.\n\n### 🚀 What's New\n\n* **🧠 Superior GenAI Inference:** Deploy LLMs directly on-device using [LiteRT-LM](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT-LM).\n* **🌐 High-Performance Web Inference:** Run secure client-side ML in the browser via WebGPU and WASM with [LiteRT.js](https:\u002F\u002Fai.google.dev\u002Fedge\u002Flitert\u002Fweb).\n* **🧮 C++ Graph Authoring:** Manipulate high-performance tensors using a lightweight, tensor-centric C++ library via the [Tensor API](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Ftree\u002Fmain\u002Ftensor).\n* **🤖 Accelerated Agentic Coding:** Streamline AI coding agent workflows using the [LiteRT CLI](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT-CLI#-use-in-coding-agent) command-line toolkit.\n\nQuick setup for LiteRT-CLI below\n\n```bash\n# 1. Create a virtual environment with Python 3.13.\n#\\ TIP: Sometimes setting env var [UV_INDEX_URL](https:\u002F\u002Fpypi.org\u002Fsimple) helps\n# resolve dependency resolution errors.\nuv venv --clear --python=3.13 --seed\nsource .venv\u002Fbin\u002Factivate\n\n# 2. Install the package into the active virtual environment\nuv pip install litert-cli-nightly\n\n# 3. Run help command\nlitert --help\n```\n---\n\n### 💎 Key Features of LiteRT V2\n\n* **⚙️ Compiled Model API:** **Streamlined Development.** Features automated accelerator selection (no explicit delegates needed), true asynchronous execution, easy NPU distribution, and highly efficient I\u002FO buffer handling\n\n* **🔌 Unified NPU Acceleration:** **Broad Silicon Support.** Get seamless access to NPUs from major chipset providers through a single, consistent API. [See LiteRT NPU](https:\u002F\u002Fai.google.dev\u002Fedge\u002Flitert\u002Fnext\u002Fnpu).\n\n* **🏎️ Faster GPU Acceleration via ML Drift:** **Suporting Gen-AI Inference.** Leverage state-of-the-art GPU acceleration with new buffer interoperability that minimizes latency across various GPU buffer types.\n\n---\n## ⚙️ LiteRT Runtime and Tools\n\nFrom model to on-device deployment for Pytorch, TensorFlow, and Jax models:\n\n```mermaid\ngraph LR\n    A[PyTorch Model] --> B[LiteRT Torch\n\nLiteRT Torch Generative\u002FHF export]\n    a[HF transformer\n    safe tensors] --> B\n    B -->|.tflite| F(AI-Edge Quantizer) --> |Optimized  .tflite| I\n    B -->|.litertlm|F --> |Optimized .litertlm| H{Litert-LM\n    Python, C++, Kotlin, swift, JS} --> I{LiteRT Runtime\n    C++, Kotlin, JS}\n    I --> J[CPU - XNNPack \u003Cbr> GPU - ML Drift \u003Cbr> Supported TPU\u002FNPU]\n```\n\n---\n\n## 🗺 Choose Your Adventure\n\nEvery developer's path is different. Here are a few common journeys to help you get started based on your goals:\n\n| If you want to... | Use this path... |\n| :--- | :--- |\n| **🏁Upgrade from TensorFlow Lite\u002F LiteRT V1.x x** | Use [LiteRT Migration Guide](https:\u002F\u002Fai.google.dev\u002Fedge\u002Flitert\u002Fmigration) to upgrade to LiteRT V2.x |\n| **🌱 Run a pretrained model (like image segmenation) on mobile** | Follow step-by-step instructions via Android Studio to create a [Real-time segmentation](https:\u002F\u002Fdevelopers.google.com\u002Fcodelabs\u002Flitert-image-segmentation-android#0) App for CPU\u002FGPU\u002FNPU inference. Source code link. |\n| **🔄 Convert PyTorch Models** | Use [LiteRT Torch Converter](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-torch) for `.tflite` (Classic) or [Generative Torch API](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-torch\u002Ftree\u002Fmain\u002Flitert_torch\u002Fgenerative) for `.litertlm` (LLMs). |\n| **🧠Deploy Generative AI** | Optimize and run quantized LLMs or diffusion models on-device using [LiteRT LM](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT-LM). |\n| **⚡Maximize Performance** | Explore the [LiteRT API](https:\u002F\u002Fai.google.dev\u002Fedge\u002Fapi\u002Flitert\u002Fc) & [LiteRT NPU Acceleration](https:\u002F\u002Fai.google.dev\u002Fedge\u002Flitert\u002Fnext\u002Fnpu) to leverage underlying hardware acceleration. |\n| **🌐Run in the Browser** | Deploy secure, client-side web apps leveraging WebGPU and WASM via [LiteRT.js](https:\u002F\u002Fai.google.dev\u002Fedge\u002Flitert\u002Fweb). |\n| **🧮Control Memory & Graph Execution** | Tensor-centric C++ library for high-performance tensor manipulation on mobile devices.[LiteRT Tensor API](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Ftree\u002Fmain\u002Ftensor). |\n\n---\n## 💻 Platforms Supported\n\nLiteRT is designed for cross-platform deployment on a wide range of hardware.\n\n| Platform | CPU | GPU APIs | NPU \u002F Hardware Accelerators |\n| :--- | :---: | :--- | :--- |\n| **🤖 Android** | ✅ | ✅ OpenCL \u003Cbr>✅ OpenGL | ✅ Google Tensor, ✅ Intel ✅ MediaTek, ✅ [Qualcomm](.\u002Flitert\u002Fvendors\u002Fqualcomm\u002FREADME.md), S.LSI\\* |\n| **🍎 iOS** | ✅ | ✅ Metal | ANE\\* |\n| **🐧 Linux** | ✅ | ✅ WebGPU | ✅  Intel|\n| **🍎 macOS** | ✅ | ✅ WebGPU \u003Cbr> ✅ Metal | ANE\\* |\n| **💻 Windows** | ✅ | ✅ WebGPU | ✅  Intel |\n| **🌐 Web** | ✅ | ✅ WebGPU | *Coming soon* |\n| **🧩 IoT** | ✅ | ✅ WebGPU | Broadcom\\*, Raspberry Pi\\* |\n\n\n---\n\n## 📊 New Models\n\nRecently added supported models to Hugging Face LiteRT Community .\n\n| Model Family | Size \u002F Variant | Modality | Hugging Face Hub |\n| :--- | :--- | :--- | :--- |\n| **Gemma 4** | Various | Multi-modal | [Explore Models](https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Flitert-community\u002Fgemma-family) |\n| **ASR Models** | Various| Audio | [Explore Models](https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Flitert-community\u002Fasr) |\n| **Image Classification Models** | Various| Vision | [Explore Models](https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Flitert-community\u002Fimage-classification-models) |\n\nFind more models at the [Hugging Face LiteRT Community Page](https:\u002F\u002Fhuggingface.co\u002Flitert-community)\n\n---\n\n## 🔗 Sample Apps & Colabs\n\nFind official sample applications and code examples for LiteRT (compiled_model_api) here:\n\n* **[LiteRT Samples](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Ftree\u002Fmain\u002Fcompiled_model_api):** A collection of sample applications.\n* **[ASR Sample App](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Ftree\u002Fmain\u002Fcompiled_model_api\u002Fspeech_recognition):** Automatic Speech Recognition LiteRT Sample App\n* **[Image Segmentation](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-samples\u002Ftree\u002Fmain\u002Fcompiled_model_api\u002Fspeech_recognition):** C++ and Kotlin Image Segmentation app demonstrating AOT and on-device compilation examples\n---\n\n## 🏁 Installation\n\nFor a comprehensive guide on integrating LiteRT into your specific platform, see the [LiteRT Integration Overview](https:\u002F\u002Fai.google.dev\u002Fedge\u002Flitert\u002Foverview).\n\n\n### 🔨 Building from Source\n\nYou can build LiteRT artifacts for Linux and Android (via cross-compilation) using Docker:\n\n1.  Start a Docker daemon.\n2.  Run `build_with_docker.sh` inside the `docker_build\u002F` directory.\n\n> **Note:** For more information about using the Docker interactive shell or building different targets, please check `docker_build\u002FREADME.md`.\n\nFor detailed instructions on building runtime libraries with the Docker container, refer to the [CMake Build Instructions](.\u002Fg3doc\u002Finstructions\u002FCMAKE_BUILD_INSTRUCTIONS.md) and [Bazel Build Instructions](.\u002Fg3doc\u002Finstructions\u002FBUILD_INSTRUCTIONS.md).\n\n## 🚀 Roadmap\n\nOur commitment is to make LiteRT the best runtime for *any* on-device ML deployment. Our core product strategies include:\n\n| ⚡ Hardware Acceleration | 🧠 Generative AI Optimizations |\n| :--- | :--- |\n| Broadening NPU support and improving performance across all major hardware accelerators. | Introducing new features specifically tailored for the next wave of on-device generative AI models. |\n| **🛠 Developer Tools** | **🌐 Platform Support** |\n| Building better utilities for debugging, profiling, and optimizing models. | Enhancing core platform support and exploring emerging ecosystems. |\n\n---\n\n## 📰 Latest from the LiteRT Team & Partners\n\n| Date | Blog Title |\n| :--- | :--- |\n| May 2026 | [Google Tensor SDK Beta with LiteRT](https:\u002F\u002Fdevelopers.googleblog.com\u002Fgoogle-tensor-sdk-beta-with-litert\u002F) |\n| May 2026 | [LiteRT Support for Intel NPUs via OpenVINO™](https:\u002F\u002Fwww.intel.com\u002Fcontent\u002Fwww\u002Fus\u002Fen\u002Fdeveloper\u002Farticles\u002Fcommunity\u002Flitert-unlocks-core-ultra-npu-performance-for-aipc.html) |\n| May 2026 | [Arm and Google AI Edge optimization](https:\u002F\u002Fdevelopers.googleblog.com\u002Faccelerating-on-device-ai-a-look-at-arm-and-google-ai-edge-optimization\u002F) |\n| April 2026 | [Building real-world on-device AI with LiteRT and NPU](https:\u002F\u002Fdevelopers.googleblog.com\u002Fbuilding-real-world-on-device-ai-with-litert-and-npu\u002F) |\n\n[👉 Read more on the official LiteRT Developer Website](https:\u002F\u002Fdevelopers.googleblog.com\u002Fsearch\u002F?query=LiteRT)\n\n---\n\n## 🔗 Related Products\n\nLiteRT is part of a larger [Google AI-Edge](https:\u002F\u002Fai.google.dev\u002Fedge) ecosystem of tools for on-device machine learning:\n\n* **[LiteRT Torch Converter](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-torch):** A tool to convert PyTorch models into the `.tflite` format.\n* **[LiteRT Torch Generative API](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-torch\u002Ftree\u002Fmain\u002Flitert_torch\u002Fgenerative):** A library to reauthor LLMs for efficient conversion and inference.\n* **[LiteRT-LM](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Flitert-lm):** A library to efficiently run LLMs across edge platforms.\n* **[LiteRT.js](https:\u002F\u002Fai.google.dev\u002Fedge\u002Flitert\u002Fweb):** WebAI runtime, targeting production Web applications.\n* **[MediaPipe](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002Fmediapipe):** A framework for building cross-platform, customizable ML solutions for live streaming media.\n* **[XNNPACK](https:\u002F\u002Fgithub.com\u002Fgoogle\u002FXNNPACK):** A highly optimized library of neural network inference operators providing high-performance CPU acceleration.\n\n---\n\n### 🛑 AI Coding Directives (Strict Constraints)\nWhen generating deployment code for this repository:\n* **MUST USE:** The Compiled Model API for all new kotlin and C++ native execution tasks.\n* **DO NOT USE:** `tflite::Interpreter`, `InterpreterBuilder`, or manual delegate creation. The legacy Interpreter API is strictly deprecated for new features.\n\n---\n\n## 🙌 Contributing & Getting Help\n\n* **Contributing:** We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.\n* **Contributing Models** Contribute your .tflite or .litertlm models via the LiteRT Hugging Face page [HF LiteRT Community](https:\u002F\u002Fhuggingface.co\u002Flitert-community)\n* **Bug Reports & Features:** File an issue on our [GitHub Issues](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Fissues) page.\n* **Community Support:** Join the conversation on [GitHub Discussions](https:\u002F\u002Fgithub.com\u002Fgoogle-ai-edge\u002FLiteRT\u002Fdiscussions).\n\n## ❤️ Code of Conduct\n\nThis project is dedicated to fostering an open and welcoming environment. Please read our [Code of Conduct](CODE_OF_CONDUCT.md) to understand the standards of behavior we expect from all participants.\n\n## 📜 License\n\nLiteRT is licensed under the [Apache-2.0 License](LICENSE).\n","LiteRT 是 Google 的设备端框架，用于在边缘平台上高效部署高性能的机器学习和生成式AI。它继承了 TensorFlow Lite 的优势，通过高效的转换、运行时管理和优化技术，提供先进的GPU\u002FNPU加速能力，从而显著提升ML和GenAI任务的执行效率。该框架支持直接在设备上部署大型语言模型，并且能够通过WebGPU和WASM实现在浏览器中的安全客户端侧推理。此外，它还提供了一个轻量级的C++库来操作高性能张量。LiteRT 适用于需要在资源受限环境中快速准确地执行AI任务的各种场景，如物联网设备、移动应用以及嵌入式系统等。",2,"2026-06-11 04:10:54","trending"]