[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-221":3},{"id":4,"name":5,"fullName":6,"owner":5,"repo":5,"description":7,"homepage":8,"htmlUrl":9,"language":10,"languages":8,"totalLinesOfCode":8,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":8,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":16,"stars30d":17,"stars90d":15,"forks30d":15,"starsTrendScore":18,"compositeScore":19,"rankGlobal":8,"rankLanguage":8,"license":8,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":20,"hasPages":20,"topics":22,"createdAt":8,"pushedAt":8,"updatedAt":42,"readmeContent":43,"aiSummary":44,"trendingCount":15,"starSnapshotCount":15,"syncStatus":45,"lastSyncTime":46,"discoverSource":47},221,"kserve","kserve\u002Fkserve","Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes",null,"https:\u002F\u002Fgithub.com\u002Fkserve\u002Fkserve","Go",5562,1520,70,511,0,20,123,8,40.55,false,"main",[23,24,25,26,27,28,29,30,31,32,33,34,35,5,36,37,38,39,40,41],"knative","machine-learning","model-interpretability","model-serving","istio","kubeflow","artificial-intelligence","tensorflow","pytorch","xgboost","kubernetes","k8s","service-mesh","hacktoberfest","mlops","genai","llm-inference","cncf","vllm","2026-06-12 02:00:10","# KServe\n[![go.dev reference](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fgo.dev-reference-007d9c?logo=go&logoColor=white)](https:\u002F\u002Fpkg.go.dev\u002Fgithub.com\u002Fkserve\u002Fkserve)\n[![Go Report Card](https:\u002F\u002Fgoreportcard.com\u002Fbadge\u002Fgithub.com\u002Fkserve\u002Fkserve)](https:\u002F\u002Fgoreportcard.com\u002Freport\u002Fgithub.com\u002Fkserve\u002Fkserve)\n[![OpenSSF Best Practices](https:\u002F\u002Fbestpractices.coreinfrastructure.org\u002Fprojects\u002F6643\u002Fbadge)](https:\u002F\u002Fbestpractices.coreinfrastructure.org\u002Fprojects\u002F6643)\n[![Releases](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Frelease-pre\u002Fkserve\u002Fkserve.svg?sort=semver)](https:\u002F\u002Fgithub.com\u002Fkserve\u002Fkserve\u002Freleases)\n[![LICENSE](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flicense\u002Fkserve\u002Fkserve.svg)](https:\u002F\u002Fgithub.com\u002Fkserve\u002Fkserve\u002Fblob\u002Fmaster\u002FLICENSE)\n[![Slack Status](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fslack-join_chat-white.svg?logo=slack&style=social)](https:\u002F\u002Fgithub.com\u002Fkserve\u002Fcommunity\u002Fblob\u002Fmain\u002FREADME.md#questions-and-issues)\n[![Gurubase](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGurubase-Ask%20KServe%20Guru-006BFF)](https:\u002F\u002Fgurubase.io\u002Fg\u002Fkserve)\n\nKServe is a standardized distributed generative and predictive AI inference platform for scalable, multi-framework deployment on Kubernetes.\n\nKServe is being [used by many organizations](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fcommunity\u002Fadopters) and is a [Cloud Native Computing Foundation (CNCF)](https:\u002F\u002Fwww.cncf.io\u002F) incubating project.\n\nFor more details, visit the [KServe website](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002F).\n\n![KServe](\u002Fdocs\u002Fdiagrams\u002Fkserve_new.png)\n\n### Why KServe?\n\nSingle platform that unifies Generative and Predictive AI inference on Kubernetes. Simple enough for quick deployments, yet powerful enough to handle enterprise-scale AI workloads with advanced features.\n\n### Features\n\n**Generative AI**\n  * 🧮 **Optimized Backends**: Support for vLLM and llm-d for optimized performance for serving LLMs\n  * 📌 **Standardization**: OpenAI-compatible inference protocol for seamless integration with LLMs\n  * 🚅 **GPU Acceleration**: High-performance serving with GPU support and optimized memory management for large models\n  * 💾 **Model Caching**: Intelligent model caching to reduce loading times and improve response latency for frequently used models\n  * 🗂️ **KV Cache Offloading**: Advanced memory management with KV cache offloading to CPU\u002Fdisk for handling longer sequences efficiently\n  * 📈 **Autoscaling**: Request-based autoscaling capabilities optimized for generative workload patterns\n  * 🔧 **Hugging Face Ready**: Native support for Hugging Face models with streamlined deployment workflows\n\n**Predictive AI**\n  * 🧮 **Multi-Framework**: Support for TensorFlow, PyTorch, scikit-learn, XGBoost, ONNX, and more\n  * 🔀 **Intelligent Routing**: Seamless request routing between predictor, transformer, and explainer components with automatic traffic management\n  * 🔄 **Advanced Deployments**: Canary rollouts, inference pipelines, and ensembles with InferenceGraph\n  * ⚡ **Autoscaling**: Request-based autoscaling with scale-to-zero for predictive workloads\n  * 🔍 **Model Explainability**: Built-in support for model explanations and feature attribution to understand prediction reasoning\n  * 📊 **Advanced Monitoring**: Enables payload logging, outlier detection, adversarial detection, and drift detection\n  * 💰 **Cost Efficient**: Scale-to-zero on expensive resources when not in use, reducing infrastructure costs\n\n### Learn More\nTo learn more about KServe, how to use various supported features, and how to participate in the KServe community, \nplease follow the [KServe website documentation](https:\u002F\u002Fkserve.github.io\u002Fwebsite). \nAdditionally, we have compiled a list of [presentations and demos](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fcommunity\u002Fpresentations) to dive through various details.\n\n### :hammer_and_wrench: Installation\n\n#### Standalone Installation\n- **[Standard Kubernetes Installation](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fadmin-guide\u002Foverview#raw-kubernetes-deployment)**: Compared to Serverless Installation, this is a more **lightweight** installation. However, this option does not support canary deployment and request based autoscaling with scale-to-zero.\n- **[Knative Installation](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fadmin-guide\u002Foverview#serverless-deployment)**: KServe by default installs Knative for **serverless deployment** for InferenceService.\n- **[ModelMesh Installation](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fadmin-guide\u002Foverview#modelmesh-deployment)**: You can optionally install ModelMesh to enable **high-scale**, **high-density** and **frequently-changing model serving** use cases. \n- **[Quick Installation](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fgetting-started\u002Fquickstart-guide)**: Install KServe on your local machine.\n\n#### Kubeflow Installation\nKServe is an important addon component of Kubeflow, please learn more from the [Kubeflow KServe documentation](https:\u002F\u002Fwww.kubeflow.org\u002Fdocs\u002Fexternal-add-ons\u002Fkserve\u002Fkserve). Check out the following guides for running [on AWS](https:\u002F\u002Fawslabs.github.io\u002Fkubeflow-manifests\u002Fmain\u002Fdocs\u002Fcomponent-guides\u002Fkserve) or [on OpenShift Container Platform](https:\u002F\u002Fgithub.com\u002Fkserve\u002Fkserve\u002Fblob\u002Fmaster\u002Fdocs\u002FOPENSHIFT_GUIDE.md).\n\n### :flight_departure: [Create your first InferenceService](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fgetting-started\u002Fgenai-first-isvc)\n\n### :bulb: [Roadmap](.\u002FROADMAP.md)\n\n### :blue_book: [InferenceService API Reference](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Freference\u002Fcrd-api)\n\n### :toolbox: [Developer Guide](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fdeveloper-guide)\n\n### :writing_hand: [Contributor Guide](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fdeveloper-guide\u002Fcontribution)\n\n### :handshake: [Adopters](https:\u002F\u002Fkserve.github.io\u002Fwebsite\u002Fdocs\u002Fcommunity\u002Fadopters)\n\n### Star History\n\n[![Star History Chart](https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=kserve\u002Fkserve&type=Date)](https:\u002F\u002Fwww.star-history.com\u002F#kserve\u002Fkserve&Date)\n\n### Contributors\n\nThanks to all of our amazing contributors!\n\n\u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fkserve\u002Fkserve\u002Fgraphs\u002Fcontributors\">\n  \u003Cimg src=\"https:\u002F\u002Fcontrib.rocks\u002Fimage?repo=kserve\u002Fkserve\" \u002F>\n\u003C\u002Fa>\n","KServe是一个标准化的分布式生成式和预测性AI推理平台，支持在Kubernetes上进行可扩展的多框架部署。其核心功能包括对多种机器学习框架的支持（如TensorFlow、PyTorch、XGBoost等），以及针对生成式AI的优化后端、GPU加速、模型缓存等特性。此外，它还提供了智能路由、金丝雀发布、自动伸缩等功能来满足复杂的AI工作负载需求。KServe适合于需要高效部署和管理AI模型的企业级场景，尤其是那些希望利用Kubernetes生态优势进行大规模AI应用开发与维护的组织。",2,"2026-06-11 02:31:36","trending"]