[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-74024":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":10,"languages":10,"totalLinesOfCode":10,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":15,"forks30d":15,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":22,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":31,"readmeContent":32,"aiSummary":33,"trendingCount":15,"starSnapshotCount":15,"syncStatus":34,"lastSyncTime":35,"discoverSource":36},74024,"llm-engineer-toolkit","KalyanKS-NLP\u002Fllm-engineer-toolkit","KalyanKS-NLP","A curated list of  120+ LLM libraries category wise. ","https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fkalyanksnlp\u002F",null,10392,1652,172,7,0,4,16,34,12,89.05,"Apache License 2.0",false,"main",true,[26,27,28,29,30],"ai-engineer","generative-ai","large-language-models","llm-engineer","llms","2026-06-12 04:01:12","# 👨🏻‍💻 LLM Engineer Toolkit \nThis repository contains a curated list of 120+ LLM libraries category wise.\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fkalyanksnlp\u002F\">\n    \u003Cimg src=\"https:\u002F\u002Fcustom-icon-badges.demolab.com\u002Fbadge\u002FKalyan%20KS-0A66C2?logo=linkedin-white&logoColor=fff\" alt=\"LinkedIn\">\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fx.com\u002Fkalyan_kpl\">\n    \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FKalyan%20KS-%23000000.svg?logo=X&logoColor=white\" alt=\"Twitter\">\n  \u003C\u002Fa>\n   \u003Ca href=\"https:\u002F\u002Fwww.youtube.com\u002F@kalyanksnlp\">\n    \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FKalyan%20KS-%23FF0000.svg?logo=YouTube&logoColor=white\" alt=\"Twitter\">\n  \u003C\u002Fa>\n\t\n\u003C\u002Fp>\n\n## AI Research Workflow (Webinar)\n\nIn this webinar, you will learn how to do AI research step-by-step. [Register](https:\u002F\u002Ftopmate.io\u002Fkalyan_ksnlp\u002F2033856)\n\nThis webinar covers the different phases in AI Research workflow\n\n- Learning Research Pre-Requisites\n- Research Problem Selection\n- Research Gaps Identification\n- Research Idea Formulation\n- Research Idea Implementation\n- Conference \u002F Journal Selection\n- Research Paper Writing\n\n## Related Repositories\n- 👨🏻‍💻[LLM Interview Questions and Answers Hub](https:\u002F\u002Fgithub.com\u002FKalyanKS-NLP\u002FLLM-Interview-Questions-and-Answers-Hub) - 100+ LLM interview questions and answers (basic to advanced). \n- 🚀[Prompt Engineering Techniques Hub](https:\u002F\u002Fgithub.com\u002FKalyanKS-NLP\u002FPrompt-Engineering-Techniques-Hub)  - 25+ prompt engineering techniques with LangChain implementations.\n- 🩸[LLM, RAG and Agents Survey Papers Collection](https:\u002F\u002Fgithub.com\u002FKalyanKS-NLP\u002FLLM-Survey-Papers-Collection) - Category wise collection of 200+ survey papers.\n\n\n## Stay Updated with Generative AI, LLMs, Agents and RAG.\n\nJoin 🚀 [**AIxFunda** free newsletter](https:\u002F\u002Faixfunda.substack.com\u002F) to get *latest updates* and *interesting tutorials* related to Generative AI, LLMs, Agents and RAG. \n- ✨ Weekly GenAI updates\n- 📄 Weekly LLM, Agents and RAG paper updates\n- 📝 1 fresh blog post on an interesting topic every week\n\n## Quick links\n||||\n|---|---|---|\n| [🚀 LLM Training](#llm-training-and-fine-tuning) | [🧱 LLM Application Development](#llm-application-development) | [🩸LLM RAG](#llm-rag) | \n| [🟩 LLM Inference](#llm-inference)| [🚧 LLM Serving](#llm-serving) | [📤 LLM Data Extraction](#llm-data-extraction) |\n| [🌠 LLM Data Generation](#llm-data-generation) | [💎 LLM Agents](#llm-agents)|[⚖️ LLM Evaluation](#llm-evaluation) | \n| [🔍 LLM Monitoring](#llm-monitoring) | [📅 LLM Prompts](#llm-prompts) | [📝 LLM Structured Outputs](#llm-structured-outputs) |\n| [🛑 LLM Safety and Security](#llm-safety-and-security) | [💠 LLM Embedding Models](#llm-embedding-models) | [❇️ Others](#others) |\n\n\n\n## LLM Training and Fine-Tuning\n| Library             | Description                                                                                     | Link |\n|---------------------|-------------------------------------------------------------------------------------------------|------|\n| unsloth            | Fine-tune LLMs faster with less memory.                                                          | [Link](https:\u002F\u002Fgithub.com\u002Funslothai\u002Funsloth) |\n| PEFT                | State-of-the-art Parameter-Efficient Fine-Tuning library.                                       | [Link](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fpeft) |\n| TRL                 | Train transformer language models with reinforcement learning.                                  | [Link](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl) |\n| Transformers       | Transformers provides thousands of pretrained models to perform tasks on different modalities such as text, vision, and audio. | [Link](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftransformers) |\n| Axolotl           | Tool designed to streamline post-training for various AI models.                                 | [Link](https:\u002F\u002Fgithub.com\u002Faxolotl-ai-cloud\u002Faxolotl\u002F) |\n| LLMBox             | A comprehensive library for implementing LLMs, including a unified training pipeline and comprehensive model evaluation. | [Link](https:\u002F\u002Fgithub.com\u002FRUCAIBox\u002FLLMBox) |\n| LitGPT             | Train and fine-tune LLM lightning fast.                                                          | [Link](https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flitgpt) |\n| Mergoo            | A library for easily merging multiple LLM experts, and efficiently train the merged LLM.         | [Link](https:\u002F\u002Fgithub.com\u002FLeeroo-AI\u002Fmergoo) |\n| Llama-Factory      | Easy and efficient LLM fine-tuning.                                                              | [Link](https:\u002F\u002Fgithub.com\u002Fhiyouga\u002FLLaMA-Factory) |\n| Ludwig            | Low-code framework for building custom LLMs, neural networks, and other AI models.               | [Link](https:\u002F\u002Fgithub.com\u002Fludwig-ai\u002Fludwig) |\n| Txtinstruct       | A framework for training instruction-tuned models.                                               | [Link](https:\u002F\u002Fgithub.com\u002Fneuml\u002Ftxtinstruct) |\n| Lamini            | An integrated LLM inference and tuning platform.                                                 | [Link](https:\u002F\u002Fgithub.com\u002Flamini-ai\u002Flamini) |\n| XTuring           | xTuring provides fast, efficient and simple fine-tuning of open-source LLMs, such as Mistral, LLaMA, GPT-J, and more. | [Link](https:\u002F\u002Fgithub.com\u002Fstochasticai\u002FxTuring) |\n| RL4LMs            | A modular RL library to fine-tune language models to human preferences.                          | [Link](https:\u002F\u002Fgithub.com\u002Fallenai\u002FRL4LMs) |\n| DeepSpeed         | DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. | [Link](https:\u002F\u002Fgithub.com\u002Fdeepspeedai\u002FDeepSpeed) |\n| torchtune         | A PyTorch-native library specifically designed for fine-tuning LLMs.                             | [Link](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Ftorchtune) |\n| PyTorch Lightning | A library that offers a high-level interface for pretraining and fine-tuning LLMs.               | [Link](https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Fpytorch-lightning) |\n\n\n## LLM Application Development\n\u003Cp align = \"center\"> \u003Cb> Frameworks \u003C\u002Fb> \u003C\u002Fp>\n\n| Library        | Description                                                                                               | Link  |\n|--------------|------------------------------------------------------------------------------------------------------|-------|\n| LangChain    | LangChain is a framework for developing applications powered by large language models (LLMs).          | [Link](https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangchain) |\n| Llama Index  | LlamaIndex is a data framework for your LLM applications.                                              | [Link](https:\u002F\u002Fgithub.com\u002Frun-llama\u002Fllama_index) |\n| HayStack     | Haystack is an end-to-end LLM framework that allows you to build applications powered by LLMs, Transformer models, vector search and more. | [Link](https:\u002F\u002Fgithub.com\u002Fdeepset-ai\u002Fhaystack) |\n| Prompt flow  | A suite of development tools designed to streamline the end-to-end development cycle of LLM-based AI applications. | [Link](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fpromptflow) |\n| Griptape     | A modular Python framework for building AI-powered applications.                                        | [Link](https:\u002F\u002Fgithub.com\u002Fgriptape-ai\u002Fgriptape) |\n| Weave        | Weave is a toolkit for developing Generative AI applications.                                          | [Link](https:\u002F\u002Fgithub.com\u002Fwandb\u002Fweave) |\n| Llama Stack  | Build Llama Apps.                                                                                      | [Link](https:\u002F\u002Fgithub.com\u002Fmeta-llama\u002Fllama-stack) |\n\n\n\u003Cp align = \"center\"> \u003Cb> Data Preparation \u003C\u002Fb> \u003C\u002Fp>\n\n| Library        | Description                                                                                               | Link  |\n|--------------|------------------------------------------------------------------------------------------------------|-------|\n| Data Prep Kit | Data Prep Kit accelerates unstructured data preparation for LLM app developers. Developers can use Data Prep Kit to cleanse, transform, and enrich use case-specific unstructured data to pre-train LLMs, fine-tune LLMs, instruct-tune LLMs, or build RAG applications. | [Link](https:\u002F\u002Fgithub.com\u002Fdata-prep-kit\u002Fdata-prep-kit) | \n\n\u003Cp align = \"center\"> \u003Cb> Multi API Access \u003C\u002Fb> \u003C\u002Fp>\n\n| Library        | Description                                                                                               | Link  |\n|--------------|------------------------------------------------------------------------------------------------------|-------|\n| LiteLLM      | Library to call 100+ LLM APIs in OpenAI format.                                                        | [Link](https:\u002F\u002Fgithub.com\u002FBerriAI\u002Flitellm) |\n| AI Gateway   | A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.                                                 | [Link](https:\u002F\u002Fgithub.com\u002FPortkey-AI\u002Fgateway) |\n\n\u003Cp align = \"center\"> \u003Cb> Routers \u003C\u002Fb> \u003C\u002Fp>\n\n| Library        | Description                                                                                               | Link  |\n|--------------|------------------------------------------------------------------------------------------------------|-------|\n| RouteLLM     | Framework for serving and evaluating LLM routers - save LLM costs without compromising quality. Drop-in replacement for OpenAI's client to route simpler queries to cheaper models.                                                      | [Link](https:\u002F\u002Fgithub.com\u002Flm-sys\u002FRouteLLM) |\n\n\n\u003Cp align = \"center\"> \u003Cb> Memory \u003C\u002Fb> \u003C\u002Fp>\n\n| Library        | Description                                                                                               | Link  |\n|--------------|------------------------------------------------------------------------------------------------------|-------|\n| mem0         | The Memory layer for your AI apps.                                                                     | [Link](https:\u002F\u002Fgithub.com\u002Fmem0ai\u002Fmem0) |\n| Memoripy     | An AI memory layer with short- and long-term storage, semantic clustering, and optional memory decay for context-aware applications. | [Link](https:\u002F\u002Fgithub.com\u002Fcaspianmoon\u002Fmemoripy) |\n| Letta (MemGPT)     | An open-source framework for building stateful LLM applications with advanced reasoning capabilities and transparent long-term memory | [Link](https:\u002F\u002Fgithub.com\u002Fletta-ai\u002Fletta) |\n| Memobase     | A user profile-based memory system designed to bring long-term user memory to your Generative AI applications. | [Link](https:\u002F\u002Fgithub.com\u002Fmemodb-io\u002Fmemobase) |\n\n\u003Cp align = \"center\"> \u003Cb> Interface \u003C\u002Fb> \u003C\u002Fp>\n\n| Library        | Description                                                                                               | Link  |\n|--------------|------------------------------------------------------------------------------------------------------|-------|\n| Streamlit    | A faster way to build and share data apps. Streamlit lets you transform Python scripts into interactive web apps in minutes                                                             | [Link](https:\u002F\u002Fgithub.com\u002Fstreamlit\u002Fstreamlit) |\n| Gradio       | Build and share delightful machine learning apps, all in Python.                                       | [Link](https:\u002F\u002Fgithub.com\u002Fgradio-app\u002Fgradio) |\n| AI SDK UI    | Build chat and generative user interfaces.                                                             | [Link](https:\u002F\u002Fsdk.vercel.ai\u002Fdocs\u002Fintroduction) |\n| AI-Gradio    | Create AI apps powered by various AI providers.                                                        | [Link](https:\u002F\u002Fgithub.com\u002FAK391\u002Fai-gradio) |\n| Simpleaichat | Python package for easily interfacing with chat apps, with robust features and minimal code complexity. | [Link](https:\u002F\u002Fgithub.com\u002Fminimaxir\u002Fsimpleaichat) |\n| Chainlit     | Build production-ready Conversational AI applications in minutes.                                      | [Link](https:\u002F\u002Fgithub.com\u002FChainlit\u002Fchainlit) |\n\n\n\u003Cp align = \"center\"> \u003Cb> Low Code \u003C\u002Fb> \u003C\u002Fp>\n\n| Library        | Description                                                                                               | Link  |\n|--------------|------------------------------------------------------------------------------------------------------|-------|\n| LangFlow     | LangFlow is a low-code app builder for RAG and multi-agent AI applications. It’s Python-based and agnostic to any model, API, or database.                           | [Link](https:\u002F\u002Fgithub.com\u002Flangflow-ai\u002Flangflow) |\n\n\u003Cp align = \"center\"> \u003Cb> Cache \u003C\u002Fb> \u003C\u002Fp>\n\n| Library        | Description                                                                                               | Link  |\n|--------------|------------------------------------------------------------------------------------------------------|-------|\n| GPTCache     | A Library for Creating Semantic Cache for LLM Queries. Slash Your LLM API Costs by 10x 💰, Boost Speed by 100x. Fully integrated with LangChain and LlamaIndex.                               | [Link](https:\u002F\u002Fgithub.com\u002Fzilliztech\u002Fgptcache) |\n\n\n## LLM RAG\n\n| Library         | Description                                                                                                      | Link  |\n|---------------|----------------------------------------------------------------------------------------------------------------|-------|\n| FastGraph RAG | Streamlined and promptable Fast GraphRAG framework designed for interpretable, high-precision, agent-driven retrieval workflows. | [Link](https:\u002F\u002Fgithub.com\u002Fcirclemind-ai\u002Ffast-graphrag) |\n| Chonkie       | RAG chunking library that is lightweight, lightning-fast, and easy to use.                                      | [Link](https:\u002F\u002Fgithub.com\u002Fchonkie-ai\u002Fchonkie) |\n| RAGChecker    | A Fine-grained Framework For Diagnosing RAG.                                                                   | [Link](https:\u002F\u002Fgithub.com\u002Famazon-science\u002FRAGChecker) |\n| RAG to Riches | Build, scale, and deploy state-of-the-art Retrieval-Augmented Generation applications.                         | [Link](https:\u002F\u002Fgithub.com\u002FSciPhi-AI\u002FR2R) |\n| BeyondLLM     | Beyond LLM offers an all-in-one toolkit for experimentation, evaluation, and deployment of Retrieval-Augmented Generation (RAG) systems. | [Link](https:\u002F\u002Fgithub.com\u002Faiplanethub\u002Fbeyondllm) |\n| SQLite-Vec    | A vector search SQLite extension that runs anywhere!                                                           | [Link](https:\u002F\u002Fgithub.com\u002Fasg017\u002Fsqlite-vec) |\n| fastRAG       | fastRAG is a research framework for efficient and optimized retrieval-augmented generative pipelines, incorporating state-of-the-art LLMs and Information Retrieval. | [Link](https:\u002F\u002Fgithub.com\u002FIntelLabs\u002FfastRAG) |\n| FlashRAG      | A Python Toolkit for Efficient RAG Research.                                                                   | [Link](https:\u002F\u002Fgithub.com\u002FRUC-NLPIR\u002FFlashRAG) |\n| Llmware       | Unified framework for building enterprise RAG pipelines with small, specialized models.                        | [Link](https:\u002F\u002Fgithub.com\u002Fllmware-ai\u002Fllmware) |\n| Rerankers     | A lightweight unified API for various reranking models.                                                        | [Link](https:\u002F\u002Fgithub.com\u002FAnswerDotAI\u002Frerankers) |\n| Vectara       | Build Agentic RAG applications.                                                                                | [Link](https:\u002F\u002Fvectara.github.io\u002Fpy-vectara-agentic\u002Flatest\u002F) |\n\n\n## LLM Inference\n\n| Library         | Description                                                                                               | Link  |\n|---------------|------------------------------------------------------------------------------------------------------|-------|\n| llama.cpp   | LLM inference in C\u002FC++. | [Link](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp) | \n| Ollama | Local LLM inference. | [Link](https:\u002F\u002Fgithub.com\u002Follama\u002Follama) | \n| vLLM         | High-throughput and memory-efficient inference and serving engine for LLMs.                            | [Link](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm) |\n| TensorRT-LLM  | TensorRT-LLM is a library for optimizing Large Language Model (LLM) inference.                        | [Link](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FTensorRT-LLM) |\n| WebLLM        | High-performance In-browser LLM Inference Engine.                                                     | [Link](https:\u002F\u002Fgithub.com\u002Fmlc-ai\u002Fweb-llm) |\n| LLM Compressor | Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment. | [Link](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fllm-compressor) |\n| LightLLM      | Python-based LLM inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance. | [Link](https:\u002F\u002Fgithub.com\u002FModelTC\u002Flightllm) |\n| torchchat     | Run PyTorch LLMs locally on servers, desktop, and mobile.                                              | [Link](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Ftorchchat) |\n\n\n## LLM Serving\n\n| Library   | Description                                                              | Link  |\n|-----------|--------------------------------------------------------------------------|-------|\n| Langcorn  | Serving LangChain LLM apps and agents automagically with FastAPI.       | [Link](https:\u002F\u002Fgithub.com\u002Fmsoedov\u002Flangcorn) |\n| LitServe  | Lightning-fast serving engine for any AI model of any size. It augments FastAPI with features like batching, streaming, and GPU autoscaling.           | [Link](https:\u002F\u002Fgithub.com\u002FLightning-AI\u002FLitServe) |\n\n\n## LLM Data Extraction\n\n| Library         | Description                                                                                                                           | Link  |\n|----------------|---------------------------------------------------------------------------------------------------------------------------------------|-------|\n| Crawl4AI       | Open-source LLM Friendly Web Crawler & Scraper.                                                                                      | [Link](https:\u002F\u002Fgithub.com\u002Funclecode\u002Fcrawl4ai) |\n| ScrapeGraphAI  | A web scraping Python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). | [Link](https:\u002F\u002Fgithub.com\u002FScrapeGraphAI\u002FScrapegraph-ai) |\n| Docling        | Docling parses documents and exports them to the desired format with ease and speed.                                                  | [Link](https:\u002F\u002Fgithub.com\u002FDS4SD\u002Fdocling) |\n| Llama Parse    | GenAI-native document parser that can parse complex document data for any downstream LLM use case (RAG, agents).                     | [Link](https:\u002F\u002Fgithub.com\u002Frun-llama\u002Fllama_cloud_services) |\n| PyMuPDF4LLM    | PyMuPDF4LLM library makes it easier to extract PDF content in the format you need for LLM & RAG environments.                        | [Link](https:\u002F\u002Fpymupdf.readthedocs.io\u002Fen\u002Flatest\u002Fpymupdf4llm\u002F) |\n| Crawlee        | A web scraping and browser automation library.                                                                                         | [Link](https:\u002F\u002Fgithub.com\u002Fapify\u002Fcrawlee-python) |\n| MegaParse      | Parser for every type of document.                                                                                                    | [Link](https:\u002F\u002Fgithub.com\u002Fquivrhq\u002Fmegaparse) |\n| ExtractThinker | Document Intelligence library for LLMs.                                                                                               | [Link](https:\u002F\u002Fgithub.com\u002Fenoch3712\u002FExtractThinker) |\n\n\n## LLM Data Generation\n\n| Library       | Description                                                                                          | Link  |\n|--------------|--------------------------------------------------------------------------------------------------|-------|\n| DataDreamer  | DataDreamer is a powerful open-source Python library for prompting, synthetic data generation, and training workflows. | [Link](https:\u002F\u002Fgithub.com\u002Fdatadreamer-dev\u002FDataDreamer) |\n| fabricator   | A flexible open-source framework to generate datasets with large language models.                   | [Link](https:\u002F\u002Fgithub.com\u002FflairNLP\u002Ffabricator) |\n| Promptwright | Synthetic Dataset Generation Library.                                                               | [Link](https:\u002F\u002Fgithub.com\u002Fstacklok\u002Fpromptwright) |\n| EasyInstruct | An Easy-to-use Instruction Processing Framework for Large Language Models.                          | [Link](https:\u002F\u002Fgithub.com\u002Fzjunlp\u002FEasyInstruct) |\n\n\n## LLM Agents\n\n| Library         | Description                                                                                                 | Link  |\n|----------------|---------------------------------------------------------------------------------------------------------|-------|\n| CrewAI        | Framework for orchestrating role-playing, autonomous AI agents.                                          | [Link](https:\u002F\u002Fgithub.com\u002FcrewAIInc\u002FcrewAI) |\n| LangGraph     | Build resilient language agents as graphs.                                                               | [Link](https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flanggraph) |\n| Agno          | Build AI Agents with memory, knowledge, tools, and reasoning. Chat with them using a beautiful Agent UI.  | [Link](https:\u002F\u002Fgithub.com\u002Fagno-agi\u002Fagno) |\n| Agents SDK    | Build agentic apps using LLMs with context, tools, hand off to other specialized agents.                  | [Link](https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fguides\u002Fagents-sdk) |\n| AutoGen       | An open-source framework for building AI agent systems.                                                  | [Link](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fautogen) |\n| Smolagents    | Library to build powerful agents in a few lines of code.                                                 | [Link](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fsmolagents) |\n| Pydantic AI | Python agent framework to build production grade applications with Generative AI. | [Link](https:\u002F\u002Fai.pydantic.dev\u002F) |\n| CAMEL | Open-source multi-agent framework with various toolkits and use-cases available. | [Link](https:\u002F\u002Fgithub.com\u002Fcamel-ai\u002Fcamel) |\n| BeeAI | Build production-ready multi-agent systems in Python. | [Link](https:\u002F\u002Fgithub.com\u002Fi-am-bee\u002Fbeeai-framework\u002Ftree\u002Fmain\u002Fpython) | \n| gradio-tools  | A Python library for converting Gradio apps into tools that can be leveraged by an LLM-based agent to complete its task. | [Link](https:\u002F\u002Fgithub.com\u002Ffreddyaboulton\u002Fgradio-tools) |\n| Composio      | Production Ready Toolset for AI Agents.                                                                  | [Link](https:\u002F\u002Fgithub.com\u002FComposioHQ\u002Fcomposio) |\n| Atomic Agents | Building AI agents, atomically.                                                                         | [Link](https:\u002F\u002Fgithub.com\u002FBrainBlend-AI\u002Fatomic-agents) |\n| Memary        | Open Source Memory Layer For Autonomous Agents.                                                          | [Link](https:\u002F\u002Fgithub.com\u002Fkingjulio8238\u002FMemary) |\n| Browser Use   | Make websites accessible for AI agents.                                                                 | [Link](https:\u002F\u002Fgithub.com\u002Fbrowser-use\u002Fbrowser-use) |\n| OpenWebAgent   | An Open Toolkit to Enable Web Agents on Large Language Models.                                           | [Link](https:\u002F\u002Fgithub.com\u002FTHUDM\u002FOpenWebAgent\u002F) |\n| Lagent        | A lightweight framework for building LLM-based agents.                                                   | [Link](https:\u002F\u002Fgithub.com\u002FInternLM\u002Flagent) |\n| LazyLLM       | A Low-code Development Tool For Building Multi-agent LLMs Applications.                                  | [Link](https:\u002F\u002Fgithub.com\u002FLazyAGI\u002FLazyLLM) |\n| Swarms        | The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework.                               | [Link](https:\u002F\u002Fgithub.com\u002Fkyegomez\u002Fswarms) |\n| ChatArena     | ChatArena is a library that provides multi-agent language game environments and facilitates research about autonomous LLM agents and their social interactions. | [Link](https:\u002F\u002Fgithub.com\u002FFarama-Foundation\u002Fchatarena) |\n| Swarm         | Educational framework exploring ergonomic, lightweight multi-agent orchestration.                        | [Link](https:\u002F\u002Fgithub.com\u002Fopenai\u002Fswarm) |\n| AgentStack    | The fastest way to build robust AI agents.                                                               | [Link](https:\u002F\u002Fgithub.com\u002FAgentOps-AI\u002FAgentStack) |\n| Archgw        | Intelligent gateway for Agents.                                                                          | [Link](https:\u002F\u002Fgithub.com\u002Fkatanemo\u002Farchgw) |\n| Flow          | A lightweight task engine for building AI agents.                                                        | [Link](https:\u002F\u002Fgithub.com\u002Flmnr-ai\u002Fflow) |\n| AgentOps      | Python SDK for AI agent monitoring.                                                                      | [Link](https:\u002F\u002Fgithub.com\u002FAgentOps-AI\u002Fagentops) |\n| Langroid      | Multi-Agent framework.                                                                                   | [Link](https:\u002F\u002Fgithub.com\u002Flangroid\u002Flangroid) |\n| Agentarium    | Framework for creating and managing simulations populated with AI-powered agents.                        | [Link](https:\u002F\u002Fgithub.com\u002FThytu\u002FAgentarium) |\n| Upsonic       | Reliable AI agent framework that supports MCP.                                                          | [Link](https:\u002F\u002Fgithub.com\u002Fupsonic\u002Fupsonic) |\n\n\n## LLM Evaluation\n\n| Library     | Description                                                                                                         | Link  |\n|------------|-----------------------------------------------------------------------------------------------------------------|-------|\n| Ragas      | Ragas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications.            | [Link](https:\u002F\u002Fgithub.com\u002Fexplodinggradients\u002Fragas) |\n| Giskard    | Open-Source Evaluation & Testing for ML & LLM systems.                                                           | [Link](https:\u002F\u002Fgithub.com\u002FGiskard-AI\u002Fgiskard) |\n| DeepEval | LLM Evaluation Framework | [Link](https:\u002F\u002Fgithub.com\u002Fconfident-ai\u002Fdeepeval) |\n| Lighteval  | All-in-one toolkit for evaluating LLMs.                                                                         | [Link](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Flighteval) |\n| Trulens | Evaluation and Tracking for LLM Experiments | [Link](https:\u002F\u002Fgithub.com\u002Ftruera\u002Ftrulens) | \n| PromptBench | A unified evaluation framework for large language models.                                                        | [Link](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fpromptbench) |\n| LangTest   | Deliver Safe & Effective Language Models. 60+ Test Types for Comparing LLM & NLP Models on Accuracy, Bias, Fairness, Robustness & More. | [Link](https:\u002F\u002Fgithub.com\u002FJohnSnowLabs\u002Flangtest) |\n| EvalPlus   | A rigorous evaluation framework for LLM4Code.                                                                    | [Link](https:\u002F\u002Fgithub.com\u002Fevalplus\u002Fevalplus) |\n| FastChat   | An open platform for training, serving, and evaluating large language model-based chatbots.                      | [Link](https:\u002F\u002Fgithub.com\u002Flm-sys\u002FFastChat) |\n| judges     | A small library of LLM judges.                                                                                   | [Link](https:\u002F\u002Fgithub.com\u002Fquotient-ai\u002Fjudges) |\n| Evals      | Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.            | [Link](https:\u002F\u002Fgithub.com\u002Fopenai\u002Fevals) |\n| AgentEvals | Evaluators and utilities for evaluating the performance of your agents.                                         | [Link](https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Fagentevals) |\n| LLMBox     | A comprehensive library for implementing LLMs, including a unified training pipeline and comprehensive model evaluation. | [Link](https:\u002F\u002Fgithub.com\u002FRUCAIBox\u002FLLMBox) |\n| Opik       | An open-source end-to-end LLM Development Platform which also includes LLM evaluation.                           | [Link](https:\u002F\u002Fgithub.com\u002Fcomet-ml\u002Fopik) |\n| PydanticAI Evals | A powerful evaluation framework designed to help you systematically evaluate the performance of LLM applications. | [Link](https:\u002F\u002Fai.pydantic.dev\u002Fevals\u002F) |\n| UQLM | A Python package for generation-time, zero-resource LLM hallucination using state-of-the-art uncertainty quantification techniques. | [Link](https:\u002F\u002Fgithub.com\u002Fcvs-health\u002Fuqlm) |\n\n\n\n## LLM Monitoring\n\n| Library              | Description                                                                                       | Link  |\n|----------------------|-------------------------------------------------------------------------------------------------|-------|\n| MLflow              | An open-source end-to-end MLOps\u002FLLMOps Platform for tracking, evaluating, and monitoring LLM applications.     | [Link](https:\u002F\u002Fgithub.com\u002Fmlflow\u002Fmlflow) |\n| Opik                | An open-source end-to-end LLM Development Platform which also includes LLM monitoring.          | [Link](https:\u002F\u002Fgithub.com\u002Fcomet-ml\u002Fopik) |\n| LangSmith           | Provides tools for logging, monitoring, and improving your LLM applications.                     | [Link](https:\u002F\u002Fgithub.com\u002Flangchain-ai\u002Flangsmith-sdk) |\n| Weights & Biases (W&B) | W&B provides features for tracking LLM performance.                                          | [Link](https:\u002F\u002Fgithub.com\u002Fwandb) |\n| Helicone            | Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. | [Link](https:\u002F\u002Fgithub.com\u002FHelicone\u002Fhelicone) |\n| Evidently          | An open-source ML and LLM observability framework.                                                | [Link](https:\u002F\u002Fgithub.com\u002Fevidentlyai\u002Fevidently) |\n| Phoenix            | An open-source AI observability platform designed for experimentation, evaluation, and troubleshooting. | [Link](https:\u002F\u002Fgithub.com\u002FArize-ai\u002Fphoenix) |\n| Observers          | A Lightweight Library for AI Observability.                                                       | [Link](https:\u002F\u002Fgithub.com\u002Fcfahlgren1\u002Fobservers) |\n\n\n## LLM Prompts\n\n| Library             | Description                                                                                                      | Link  |\n|---------------------|----------------------------------------------------------------------------------------------------------------|-------|\n| PCToolkit          | A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models.                                   | [Link](https:\u002F\u002Fgithub.com\u002F3DAgentWorld\u002FToolkit-for-Prompt-Compression) |\n| Selective Context  | Selective Context compresses your prompt and context to allow LLMs (such as ChatGPT) to process 2x more content. | [Link](https:\u002F\u002Fpypi.org\u002Fproject\u002Fselective-context\u002F) |\n| LLMLingua          | Library for compressing prompts to accelerate LLM inference.                                                  | [Link](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FLLMLingua) |\n| betterprompt       | Test suite for LLM prompts before pushing them to production.                                                 | [Link](https:\u002F\u002Fgithub.com\u002Fstjordanis\u002Fbetterprompt) |\n| Promptify         | Solve NLP Problems with LLMs & easily generate different NLP Task prompts for popular generative models like GPT, PaLM, and more with Promptify. | [Link](https:\u002F\u002Fgithub.com\u002Fpromptslab\u002FPromptify) |\n| PromptSource      | PromptSource is a toolkit for creating, sharing, and using natural language prompts.                          | [Link](https:\u002F\u002Fpypi.org\u002Fproject\u002Fpromptsource\u002F) |\n| DSPy              | DSPy is the open-source framework for programming—rather than prompting—language models.                      | [Link](https:\u002F\u002Fgithub.com\u002Fstanfordnlp\u002Fdspy) |\n| Py-priompt        | Prompt design library.                                                                                        | [Link](https:\u002F\u002Fgithub.com\u002Fzenbase-ai\u002Fpy-priompt) |\n| Promptimizer      | Prompt optimization library.                                                                                  | [Link](https:\u002F\u002Fgithub.com\u002Fhinthornw\u002Fpromptimizer) |\n\n\n## LLM Structured Outputs\n| Library |\tDescription |\tLink |\n|------------|--------------------------------------------------------|------|\n|Instructor |\tPython library for working with structured outputs from large language models (LLMs). Built on top of Pydantic, it provides a simple, transparent, and user-friendly API. | [Link](https:\u002F\u002Fgithub.com\u002Finstructor-ai\u002Finstructor) |\n| XGrammar   | An open-source library for efficient, flexible, and portable structured generation. | [Link](https:\u002F\u002Fgithub.com\u002Fmlc-ai\u002Fxgrammar) |\n| Outlines   | Robust (structured) text generation | [Link](https:\u002F\u002Fgithub.com\u002Fdottxt-ai\u002Foutlines) |\n| Guidance   | Guidance is an efficient programming paradigm for steering language models. | [Link](https:\u002F\u002Fgithub.com\u002Fguidance-ai\u002Fguidance) |\n| LMQL      | A language for constraint-guided and efficient LLM programming. | [Link](https:\u002F\u002Fgithub.com\u002Feth-sri\u002Flmql) |\n| Jsonformer | A Bulletproof Way to Generate Structured JSON from Language Models. | [Link](https:\u002F\u002Fgithub.com\u002F1rgs\u002Fjsonformer) |\n\n\n## LLM Safety and Security\n| Library         | Description  | Link |\n|---------------|-----------------------------------------------------------|------|\n| JailbreakEval | A collection of automated evaluators for assessing jailbreak attempts. | [Link](https:\u002F\u002Fgithub.com\u002FThuCCSLab\u002FJailbreakEval) |\n| EasyJailbreak | An easy-to-use Python framework to generate adversarial jailbreak prompts. | [Link](https:\u002F\u002Fgithub.com\u002FEasyJailbreak\u002FEasyJailbreak) |\n| Guardrails    | Adding guardrails to large language models. | [Link](https:\u002F\u002Fgithub.com\u002Fguardrails-ai\u002Fguardrails) |\n| LLM Guard     | The Security Toolkit for LLM Interactions. | [Link](https:\u002F\u002Fgithub.com\u002Fprotectai\u002Fllm-guard) |\n| AuditNLG      | AuditNLG is an open-source library that can help reduce the risks associated with using generative AI systems for language. | [Link](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FAuditNLG) |\n| NeMo Guardrails | NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems. | [Link](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FNeMo-Guardrails) |\n| Garak        | LLM vulnerability scanner | [Link](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fgarak) |\n| DeepTeam | The LLM Red Teaming Framework | [Link](https:\u002F\u002Fgithub.com\u002Fconfident-ai\u002Fdeepteam)|\n\n\n## LLM Embedding Models\n| Library                   | Description                                         | Link |\n|---------------------------|-----------------------------------------------------|------|\n| Sentence-Transformers     | State-of-the-Art Text Embeddings                   | [Link](https:\u002F\u002Fgithub.com\u002FUKPLab\u002Fsentence-transformers) |\n| Model2Vec                | Fast State-of-the-Art Static Embeddings             | [Link](https:\u002F\u002Fgithub.com\u002FMinishLab\u002Fmodel2vec) |\n| Text Embedding Inference | A blazing fast inference solution for text embeddings models. TEI enables high-performance extraction for the most popular models, including FlagEmbedding, Ember, GTE and E5. | [Link](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftext-embeddings-inference) |\n\n\n## Others\n| Library                 | Description  | Link |\n|-------------------------|----------------------------------------------------------------------------------------------------------------------------------|------|\n| Text Machina           | A modular and extensible Python framework, designed to aid in the creation of high-quality, unbiased datasets to build robust models for MGT-related tasks such as detection, attribution, and boundary detection. | [Link](https:\u002F\u002Fgithub.com\u002FGenaios\u002FTextMachina) |\n| LLM Reasoners          | A library for advanced large language model reasoning. | [Link](https:\u002F\u002Fgithub.com\u002Fmaitrix-org\u002Fllm-reasoners) |\n| EasyEdit               | An Easy-to-use Knowledge Editing Framework for Large Language Models. | [Link](https:\u002F\u002Fgithub.com\u002Fzjunlp\u002FEasyEdit) |\n| CodeTF                 | CodeTF: One-stop Transformer Library for State-of-the-art Code LLM. | [Link](https:\u002F\u002Fgithub.com\u002Fsalesforce\u002FCodeTF) |\n| spacy-llm              | This package integrates Large Language Models (LLMs) into spaCy, featuring a modular system for fast prototyping and prompting, and turning unstructured responses into robust outputs for various NLP tasks. | [Link](https:\u002F\u002Fgithub.com\u002Fexplosion\u002Fspacy-llm) |\n| pandas-ai              | Chat with your database (SQL, CSV, pandas, polars, MongoDB, NoSQL, etc.). | [Link](https:\u002F\u002Fgithub.com\u002FSinaptik-AI\u002Fpandas-ai) |\n| LLM Transparency Tool  | An open-source interactive toolkit for analyzing internal workings of Transformer-based language models. | [Link](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fllm-transparency-tool) |\n| Vanna                  | Chat with your SQL database. Accurate Text-to-SQL Generation via LLMs using RAG. | [Link](https:\u002F\u002Fgithub.com\u002Fvanna-ai\u002Fvanna) |\n| mergekit               | Tools for merging pretrained large language models. | [Link](https:\u002F\u002Fgithub.com\u002Farcee-ai\u002FMergeKit) |\n| MarkLLM                | An Open-Source Toolkit for LLM Watermarking. | [Link](https:\u002F\u002Fgithub.com\u002FTHU-BPM\u002FMarkLLM) |\n| LLMSanitize            | An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs). | [Link](https:\u002F\u002Fgithub.com\u002Fntunlp\u002FLLMSanitize) |\n| Annotateai             | Automatically annotate papers using LLMs. | [Link](https:\u002F\u002Fgithub.com\u002Fneuml\u002Fannotateai) |\n| LLM Reasoner          | Make any LLM think like OpenAI o1 and DeepSeek R1. | [Link](https:\u002F\u002Fgithub.com\u002Fharishsg993010\u002FLLM-Reasoner) |\n\n\n## ⭐️ Star History\n\n[![Star History Chart](https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=KalyanKS-NLP\u002Fllm-engineer-toolkit&type=Date)](https:\u002F\u002Fstar-history.com\u002F#)\n\nPlease consider giving a star, if you find this repository useful. \n\n","该项目是一个精心整理的大规模语言模型（LLM）库列表，按类别收录了超过120个相关资源。其核心功能在于为AI工程师和研究人员提供一个全面的工具集，涵盖了从LLM训练与微调、应用开发到数据提取等多个方面。技术特点包括但不限于支持各种主流框架及服务，能够帮助用户快速定位适合特定任务的最佳实践或解决方案。适用于需要利用LLM进行研究、开发新产品或优化现有系统的场景中，无论是初学者还是经验丰富的专业人士都能从中受益。",2,"2026-06-11 03:48:27","high_star"]