[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-70937":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":23,"hasPages":23,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":29,"readmeContent":30,"aiSummary":31,"trendingCount":16,"starSnapshotCount":16,"syncStatus":32,"lastSyncTime":33,"discoverSource":34},70937,"ragas","vibrantlabsai\u002Fragas","vibrantlabsai","Supercharge Your LLM Application Evaluations 🚀","https:\u002F\u002Fdocs.ragas.io",null,"Python",14326,1479,52,317,0,65,160,450,195,119.51,"Apache License 2.0",false,"main",[26,27,28],"evaluation","llm","llmops","2026-06-12 04:00:58","\u003Ch1 align=\"center\">\n  \u003Cimg style=\"vertical-align:middle\" height=\"200\"\n  src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fvibrantlabsai\u002Fragas\u002Fmain\u002Fdocs\u002F_static\u002Fimgs\u002Flogo.png\">\n\u003C\u002Fh1>\n\u003Cp align=\"center\">\n  \u003Ci>Supercharge Your LLM Application Evaluations 🚀\u003C\u002Fi>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Freleases\">\n        \u003Cimg alt=\"Latest release\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Frelease\u002Fvibrantlabsai\u002Fragas.svg\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fwww.python.org\u002F\">\n        \u003Cimg alt=\"Made with Python\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FMade%20with-Python-1f425f.svg?color=purple\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fblob\u002Fmaster\u002FLICENSE\">\n        \u003Cimg alt=\"License Apache-2.0\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flicense\u002Fvibrantlabsai\u002Fragas.svg?color=green\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fragas\u002F\">\n        \u003Cimg alt=\"Ragas Downloads per month\" src=\"https:\u002F\u002Fstatic.pepy.tech\u002Fbadge\u002Fragas\u002Fmonth\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002F5djav8GGNZ\">\n        \u003Cimg alt=\"Join Ragas community on Discord\" src=\"https:\u002F\u002Fimg.shields.io\u002Fdiscord\u002F1119637219561451644\">\n    \u003C\u002Fa>\n    \u003Ca target=\"_blank\" href=\"https:\u002F\u002Fdeepwiki.com\u002Fvibrantlabsai\u002Fragas\">\n      \u003Cimg \n        src=\"https:\u002F\u002Fdevin.ai\u002Fassets\u002Fdeepwiki-badge.png\" \n        alt=\"Ask DeepWiki.com\" \n        height=\"20\" \n      \u002F>\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Ch4 align=\"center\">\n    \u003Cp>\n        \u003Ca href=\"https:\u002F\u002Fdocs.ragas.io\u002F\">Documentation\u003C\u002Fa> |\n        \u003Ca href=\"#fire-quickstart\">Quick start\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002F5djav8GGNZ\">Join Discord\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fblog.ragas.io\u002F\">Blog\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fnewsletter.ragas.io\u002F\">NewsLetter\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fwww.ragas.io\u002Fcareers\">Careers\u003C\u002Fa>\n    \u003Cp>\n\u003C\u002Fh4>\n\nObjective metrics, intelligent test generation, and data-driven insights for LLM apps\n\nRagas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications. Say goodbye to time-consuming, subjective assessments and hello to data-driven, efficient evaluation workflows.\nDon't have a test dataset ready? We also do production-aligned test set generation.\n\n## Key Features\n\n- 🎯 Objective Metrics: Evaluate your LLM applications with precision using both LLM-based and traditional metrics.\n- 🧪 Test Data Generation: Automatically create comprehensive test datasets covering a wide range of scenarios.\n- 🔗 Seamless Integrations: Works flawlessly with popular LLM frameworks like LangChain and major observability tools.\n- 📊 Build feedback loops: Leverage production data to continually improve your LLM applications.\n\n## :shield: Installation\n\nPypi:\n\n```bash\npip install ragas\n```\n\nAlternatively, from source:\n\n```bash\npip install git+https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\n```\n\n## :fire: Quickstart\n\n### Clone a Complete Example Project\n\nThe fastest way to get started is to use the `ragas quickstart` command:\n\n```bash\n# List available templates\nragas quickstart\n\n# Create a RAG evaluation project\nragas quickstart rag_eval\n\n# Specify where you want to create it.\nragas quickstart rag_eval -o .\u002Fmy-project\n```\n\nAvailable templates:\n- `rag_eval` - Evaluate RAG systems\n\nComing Soon:\n- `agent_evals` - Evaluate AI agents\n- `benchmark_llm` - Benchmark and compare LLMs\n- `prompt_evals` - Evaluate prompt variations\n- `workflow_eval` - Evaluate complex workflows\n\n### Evaluate your LLM App\n\n`ragas` comes with pre-built metrics for common evaluation tasks. For example, Aspect Critique evaluates any aspect of your output using `DiscreteMetric`:\n\n```python\nimport asyncio\nfrom openai import AsyncOpenAI\nfrom ragas.metrics import DiscreteMetric\nfrom ragas.llms import llm_factory\n\n# Setup your LLM\nclient = AsyncOpenAI()\nllm = llm_factory(\"gpt-4o\", client=client)\n\n# Create a custom aspect evaluator\nmetric = DiscreteMetric(\n    name=\"summary_accuracy\",\n    allowed_values=[\"accurate\", \"inaccurate\"],\n    prompt=\"\"\"Evaluate if the summary is accurate and captures key information.\n\nResponse: {response}\n\nAnswer with only 'accurate' or 'inaccurate'.\"\"\"\n)\n\n# Score your application's output\nasync def main():\n    score = await metric.ascore(\n        llm=llm,\n        response=\"The summary of the text is...\"\n    )\n    print(f\"Score: {score.value}\")  # 'accurate' or 'inaccurate'\n    print(f\"Reason: {score.reason}\")\n\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n> **Note**: Make sure your `OPENAI_API_KEY` environment variable is set.\n\nFind the complete [Quickstart Guide](https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Flatest\u002Fgetstarted\u002Fquickstart)\n\n## Want help in improving your AI application using evals?\n\nIn the past 2 years, we have seen and helped improve many AI applications using evals. If you want help with improving and scaling up your AI application using evals.\n\n🔗 Book a [slot](https:\u002F\u002Fcal.com\u002Fteam\u002Fvibrantlabs\u002Fapp) or drop us a line: [founders@vibrantlabs.com](mailto:founders@vibrantlabs.com).\n\n## 🫂 Community\n\nIf you want to get more involved with Ragas, check out our [discord server](https:\u002F\u002Fdiscord.gg\u002F5qGUJ6mh7C). It's a fun community where we geek out about LLM, Retrieval, Production issues, and more.\n\n## Contributors\n\n```yml\n+----------------------------------------------------------------------------+\n|     +----------------------------------------------------------------+     |\n|     | Developers: Those who built with `ragas`.                      |     |\n|     | (You have `import ragas` somewhere in your project)            |     |\n|     |     +----------------------------------------------------+     |     |\n|     |     | Contributors: Those who make `ragas` better.       |     |     |\n|     |     | (You make PR to this repo)                         |     |     |\n|     |     +----------------------------------------------------+     |     |\n|     +----------------------------------------------------------------+     |\n+----------------------------------------------------------------------------+\n```\n\nWe welcome contributions from the community! Whether it's bug fixes, feature additions, or documentation improvements, your input is valuable.\n\n1. Fork the repository\n2. Create your feature branch (git checkout -b feature\u002FAmazingFeature)\n3. Commit your changes (git commit -m 'Add some AmazingFeature')\n4. Push to the branch (git push origin feature\u002FAmazingFeature)\n5. Open a Pull Request\n\n## 🔍 Open Analytics\n\nAt Ragas, we believe in transparency. We collect minimal, anonymized usage data to improve our product and guide our development efforts.\n\n✅ No personal or company-identifying information\n\n✅ Open-source data collection [code](.\u002Fsrc\u002Fragas\u002F_analytics.py)\n\n✅ Publicly available aggregated [data](https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas\u002Fissues\u002F49)\n\nTo opt-out, set the `RAGAS_DO_NOT_TRACK` environment variable to `true`.\n\n### Cite Us\n\n```\n@misc{ragas2024,\n  author       = {VibrantLabs},\n  title        = {Ragas: Supercharge Your LLM Application Evaluations},\n  year         = {2024},\n  howpublished = {\\url{https:\u002F\u002Fgithub.com\u002Fvibrantlabsai\u002Fragas}},\n}\n```\n","Ragas 是一个用于评估和优化大型语言模型（LLM）应用程序的工具包。它提供了客观指标、智能测试生成和数据驱动的洞察力，帮助用户告别耗时且主观的评估方式。其核心功能包括基于LLM和传统方法的精准评估指标、自动创建全面覆盖各种场景的测试数据集以及与LangChain等流行框架无缝集成。此外，Ragas还支持利用生产数据构建反馈循环，以持续改进LLM应用。该工具非常适合需要高效准确地评估和迭代自然语言处理模型质量的研发团队使用。",2,"2026-06-11 03:35:01","high_star"]