[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-9653":3},{"id":4,"name":5,"fullName":6,"owner":5,"repo":5,"description":7,"homepage":8,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":15,"forks30d":15,"starsTrendScore":19,"compositeScore":20,"rankGlobal":9,"rankLanguage":9,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":22,"topics":25,"createdAt":9,"pushedAt":9,"updatedAt":46,"readmeContent":47,"aiSummary":48,"trendingCount":15,"starSnapshotCount":15,"syncStatus":49,"lastSyncTime":50,"discoverSource":51},9653,"cleanlab","cleanlab\u002Fcleanlab","Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.","https:\u002F\u002Fcleanlab.ai",null,"Python",11508,899,86,73,0,6,11,51,21,43.86,"Apache License 2.0",false,"master",true,[26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45],"active-learning","annotation","anomaly-detection","data-annotation","data-centric-ai","data-cleaning","data-curation","data-labeling","data-profiling","data-quality","data-science","data-validation","datasets","exploratory-data-analysis","labeling","machine-learning","noisy-labels","out-of-distribution-detection","outlier-detection","weak-supervision","2026-06-12 02:02:10","\u003Cdiv align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fcleanlab\u002Fassets\u002Fmaster\u002Fcleanlab\u002Fcleanlab_logo_open_source.png\" width=60%>\n\u003C\u002Fdiv>\n\n\u003Cdiv align=\"center\">\n\u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fpypi\u002Fcleanlab\u002F\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fv\u002Fcleanlab.svg\" alt=\"pypi_versions\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fpypi\u002Fcleanlab\u002F\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpython-3.10%2B-blue\" alt=\"py_versions\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fapp.codecov.io\u002Fgh\u002Fcleanlab\u002Fcleanlab\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Fcodecov.io\u002Fgh\u002Fcleanlab\u002Fcleanlab\u002Fbranch\u002Fmaster\u002Fgraph\u002Fbadge.svg\" alt=\"coverage\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fcleanlab\u002Fstargazers\u002F\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002Fcleanlab\u002Fcleanlab?style=social&maxAge=2592000\" alt=\"Github Stars\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Ftwitter.com\u002FCleanlabAI\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Ftwitter\u002Ffollow\u002FCleanlabAI?style=social\" alt=\"Twitter\">\u003C\u002Fa>\n\u003C\u002Fdiv>\n\n\u003Ch4 align=\"center\">\n    \u003Cp>\n        \u003Ca href=\"https:\u002F\u002Fdocs.cleanlab.ai\u002F\">Documentation\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fexamples\">Examples\u003C\u002Fa> |\n        \u003Ca href=\"https:\u002F\u002Fcleanlab.ai\u002Fblog\u002Flearn\u002F\">Blog\u003C\u002Fa> |\n        \u003Ca href=\"#citation-and-related-publications\">Research\u003C\u002Fa>\n    \u003Cp>\n\u003C\u002Fh4>\n\nCleanlab’s open-source library helps you **clean** data and **lab**els by automatically detecting issues in a ML dataset. To facilitate **machine learning with messy, real-world data**, this data-centric AI package uses your *existing* models to estimate dataset problems that can be fixed to train even *better* models.\n \n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fcleanlab\u002Fassets\u002Fmaster\u002Fcleanlab\u002Fdatalab_issues.png\" width=74%>\n\u003C\u002Fp>\n\u003Cp align=\"center\">\n    Examples of various issues in Cat\u002FDog dataset \u003Cb>automatically detected\u003C\u002Fb> by cleanlab via this code:    \n\u003C\u002Fp>\n\n```python\n        lab = cleanlab.Datalab(data=dataset, label=\"column_name_for_labels\")\n        # Fit any ML model, get its feature_embeddings & pred_probs for your data\n        lab.find_issues(features=feature_embeddings, pred_probs=pred_probs)\n        lab.report()\n```\n\n- Use cleanlab to automatically check every: [text](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fdatalab\u002Ftext.html), [audio](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fdatalab\u002Faudio.html), [image](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fdatalab\u002Fimage.html), or [tabular](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fdatalab\u002Ftabular.html) dataset.\n- Use cleanlab to automatically: [detect data issues (outliers, duplicates, label errors, etc)](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fdatalab\u002Fdatalab_quickstart.html), [train robust models](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Findepth_overview.html), [infer consensus + annotator-quality for multi-annotator data](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fmultiannotator.html), [suggest data to (re)label next (active learning)](https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fexamples\u002Fblob\u002Fmaster\u002Factive_learning_multiannotator\u002Factive_learning.ipynb).\n\n\n---\n\n\n## Run cleanlab open-source\n\nThis cleanlab package runs on Python 3.10+ and supports Linux, macOS, as well as Windows.\n\n- Get started [here](https:\u002F\u002Fdocs.cleanlab.ai\u002F)! Install via `uv`, `pip`, or `conda`.\n- Developers who install the bleeding-edge from source should refer to [this master branch documentation](https:\u002F\u002Fdocs.cleanlab.ai\u002Fmaster\u002Findex.html).\n\n**Practicing data-centric AI can look like this:**\n1. Train initial ML model on original dataset.\n2. Utilize this model to diagnose data issues (via cleanlab methods) and improve the dataset.\n3. Train the same model on the improved dataset. \n4. Try various modeling techniques to further improve performance.\n\nMost folks jump from Step 1 → 4, but you may achieve big gains without *any* change to your modeling code by using cleanlab!\nContinuously boost performance by iterating Steps 2 → 4 (and try to evaluate with *cleaned* data).\n\n![](https:\u002F\u002Fraw.githubusercontent.com\u002Fcleanlab\u002Fassets\u002Fmaster\u002Fcleanlab\u002Fflowchart.png)\n\n\n## Use cleanlab with any model and in most ML tasks\n\nAll features of cleanlab work with **any dataset** and **any model**. Yes, any model: PyTorch, Tensorflow, Keras, JAX, HuggingFace, OpenAI, XGBoost, scikit-learn, etc.\n\ncleanlab is useful across a wide variety of Machine Learning tasks. Specific tasks this data-centric AI package offers dedicated functionality for include:\n1. [Binary and multi-class classification](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Findepth_overview.html)\n2. [Multi-label classification](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fmultilabel_classification.html) (e.g. image\u002Fdocument tagging)\n3. [Token classification](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Ftoken_classification.html) (e.g. entity recognition in text)\n4. [Regression](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fregression.html) (predicting numerical column in a dataset)\n5. [Image segmentation](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fsegmentation.html) (images with per-pixel annotations)\n6. [Object detection](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fobject_detection.html) (images with bounding box annotations)\n7. [Classification with data labeled by multiple annotators](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Fmultiannotator.html)\n8. [Active learning with multiple annotators](https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fexamples\u002Fblob\u002Fmaster\u002Factive_learning_multiannotator\u002Factive_learning.ipynb) (suggest which data to label or re-label to improve model most)\n9. [Outlier detection](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Foutliers.html) (identify atypical data that appears out of distribution)\n\nFor other ML tasks, cleanlab can still help you improve your dataset if appropriately applied.\nSee our [Example Notebooks](https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fexamples) and [Blog](https:\u002F\u002Fcleanlab.ai\u002Fblog\u002Flearn\u002F).\n\n\n## So fresh, so cleanlab\n\nBeyond automatically catching [all sorts of issues](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Fcleanlab\u002Fdatalab\u002Fguide\u002Fissue_type_description.html) lurking in your data, this data-centric AI package helps you deal with **noisy labels** and train more **robust ML models**.\nHere's an example:\n\n```python\n\n# cleanlab works with **any classifier**. Yup, you can use PyTorch\u002FTensorFlow\u002FOpenAI\u002FXGBoost\u002Fetc.\ncl = cleanlab.classification.CleanLearning(sklearn.YourFavoriteClassifier())\n\n# cleanlab finds data and label issues in **any dataset**... in ONE line of code!\nlabel_issues = cl.find_label_issues(data, labels)\n\n# cleanlab trains a robust version of your model that works more reliably with noisy data.\ncl.fit(data, labels)\n\n# cleanlab estimates the predictions you would have gotten if you had trained with *no* label issues.\ncl.predict(test_data)\n\n# A universal data-centric AI tool, cleanlab quantifies class-level issues and overall data quality, for any dataset.\ncleanlab.dataset.health_summary(labels, confident_joint=cl.confident_joint)\n```\n\ncleanlab **clean**s your data's **lab**els via state-of-the-art *confident learning* algorithms, published in this [paper](https:\u002F\u002Fjair.org\u002Findex.php\u002Fjair\u002Farticle\u002Fview\u002F12125) and [blog](https:\u002F\u002Fl7.curtisnorthcutt.com\u002Fconfident-learning). See some of the datasets cleaned with cleanlab at [labelerrors.com](https:\u002F\u002Flabelerrors.com).\n\ncleanlab is:\n\n1. **backed by theory** -- with [provable guarantees](https:\u002F\u002Farxiv.org\u002Fabs\u002F1911.00068) of exact label noise estimation, even with imperfect models.\n2. **fast** -- code is parallelized and scalable.\n4. **easy to use** -- one line of code to find mislabeled data, bad annotators, outliers, or train noise-robust models.\n6. **general** -- works with **[any dataset](https:\u002F\u002Flabelerrors.com\u002F)** (text, image, tabular, audio,...) + **any model** (PyTorch, OpenAI, XGBoost,...)\n\u003Cbr\u002F>\n\n![](https:\u002F\u002Fraw.githubusercontent.com\u002Fcleanlab\u002Fassets\u002Fmaster\u002Fcleanlab\u002Flabel-errors-examples.png)\n\u003Cp align=\"center\">\nExamples of incorrect given labels in various image datasets \u003Ca href=\"https:\u002F\u002Fl7.curtisnorthcutt.com\u002Flabel-errors\">found and corrected\u003C\u002Fa> using cleanlab. \nWhile these examples are from image datasets, this also works for text, audio, tabular data.\n\u003C\u002Fp>\n\n\n## Citation and related publications\n\ncleanlab is based on peer-reviewed research. Here are relevant papers to cite if you use this package:\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1911.00068\">Confident Learning (JAIR '21)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @article{northcutt2021confidentlearning,\n        title={Confident Learning: Estimating Uncertainty in Dataset Labels},\n        author={Curtis G. Northcutt and Lu Jiang and Isaac L. Chuang},\n        journal={Journal of Artificial Intelligence Research (JAIR)},\n        volume={70},\n        pages={1373--1411},\n        year={2021}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F1705.01936\">Rank Pruning (UAI '17)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{northcutt2017rankpruning,\n        author={Northcutt, Curtis G. and Wu, Tailin and Chuang, Isaac L.},\n        title={Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels},\n        booktitle = {Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence},\n        series = {UAI'17},\n        year = {2017},\n        location = {Sydney, Australia},\n        numpages = {10},\n        url = {http:\u002F\u002Fauai.org\u002Fuai2017\u002Fproceedings\u002Fpapers\u002F35.pdf},\n        publisher = {AUAI Press},\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Fjonasmueller.org\u002Finfo\u002FLabelQuality_icml.pdf\"> Label Quality Scoring (ICML '22)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{kuan2022labelquality,\n        title={Model-agnostic label quality scoring to detect real-world label errors},\n        author={Kuan, Johnson and Mueller, Jonas},\n        booktitle={ICML DataPerf Workshop},\n        year={2022}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.03920\"> Label Errors in Token Classification \u002F Entity Recognition (NeurIPS '22)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{wang2022tokenerrors,\n        title={Detecting label errors in token classification data},\n        author={Wang, Wei-Chen and Mueller, Jonas},\n        booktitle={NeurIPS Workshop on Interactive Learning for Natural Language Processing (InterNLP)},\n        year={2022}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2211.13895\"> Label Errors in Multi-Label Classification (ICLR '23)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{thyagarajan2023multilabel,\n        title={Identifying Incorrect Annotations in Multi-Label Classification Data},\n        author={Thyagarajan, Aditya and Snorrason, Elías and Northcutt, Curtis and Mueller, Jonas},\n        booktitle={ICLR Workshop on Trustworthy ML},\n        year={2023}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2309.00832\"> Label Errors in Object Detection (ICML '23)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{tkachenko2023objectlab,\n        title={ObjectLab: Automated Diagnosis of Mislabeled Images in Object Detection Data},\n        author={Tkachenko, Ulyana and Thyagarajan, Aditya and Mueller, Jonas},\n        booktitle={ICML Workshop on Data-centric Machine Learning Research},\n        year={2023}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2307.05080\"> Label Errors in Image Segmentation (ICML '23)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{lad2023segmentation,\n        title={Estimating label quality and errors in semantic segmentation data via any model},\n        author={Lad, Vedang and Mueller, Jonas},\n        booktitle={ICML Workshop on Data-centric Machine Learning Research},\n        year={2023}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.16583\"> Detecting Errors in Numerical Data (DMLR '24)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{zhou2023errors,\n        title={Detecting Errors in a Numerical Response via any Regression Model},\n        author={Zhou, Hang and Mueller, Jonas and Kumar, Mayank and Wang, Jane-Ling and Lei, Jing},\n        booktitle={Journal of Data-centric Machine Learning Research},\n        year={2024}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2207.03061\"> Out-of-Distribution Detection (ICML '22)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{kuan2022ood,\n        title={Back to the Basics: Revisiting Out-of-Distribution Detection Baselines},\n        author={Kuan, Johnson and Mueller, Jonas},\n        booktitle={ICML Workshop on Principles of Distribution Shift},\n        year={2022}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2210.06812\"> CROWDLAB for Data with Multiple Annotators (NeurIPS '22)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{goh2022crowdlab,\n        title={CROWDLAB: Supervised learning to infer consensus labels and quality scores for data with multiple annotators},\n        author={Goh, Hui Wen and Tkachenko, Ulyana and Mueller, Jonas},\n        booktitle={NeurIPS Human in the Loop Learning Workshop},\n        year={2022}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.11856\"> ActiveLab: Active learning with data re-labeling (ICLR '23)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{goh2023activelab,\n        title={ActiveLab: Active Learning with Re-Labeling by Multiple Annotators},\n        author={Goh, Hui Wen and Mueller, Jonas},\n        booktitle={ICLR Workshop on Trustworthy ML},\n        year={2023}\n    }\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\u003Csummary>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.15696\"> Detecting Dataset Drift and Non-IID Sampling (ICML '23)\u003C\u002Fa> (\u003Cb>click to show bibtex\u003C\u002Fb>) \u003C\u002Fsummary>\n\n    @inproceedings{cummings2023drift,\n        title={Detecting Dataset Drift and Non-IID Sampling via k-Nearest Neighbors},\n        author={Cummings, Jesse and Snorrason, Elías and Mueller, Jonas},\n        booktitle={ICML Workshop on Data-centric Machine Learning Research},\n        year={2023}\n    }\n\n\u003C\u002Fdetails>\n\nTo understand\u002Fcite other cleanlab functionality not described above, check out our [Blog](https:\u002F\u002Fcleanlab.ai\u002Fblog\u002Flearn).\n\n\n## Other resources\n\n- [Example Notebooks demonstrating practical applications of this package](https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fexamples)\n\n- [Cleanlab Blog](https:\u002F\u002Fcleanlab.ai\u002Fblog\u002Flearn)\n\n- [Blog post: Introduction to Confident Learning](https:\u002F\u002Fl7.curtisnorthcutt.com\u002Fconfident-learning)\n\n- [NeurIPS 2021 paper: Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https:\u002F\u002Farxiv.org\u002Fabs\u002F2103.14749)\n\n- [Introduction to Data-centric AI (MIT IAP Course)](https:\u002F\u002Fdcai.csail.mit.edu\u002F)\n\n- [Release notes for past versions](https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fcleanlab\u002Freleases)\n\n**Interested in contributing?**  See the [contributing guide](CONTRIBUTING.md), [development guide](DEVELOPMENT.md), and [ideas on useful contributions](https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fcleanlab\u002Fwiki#ideas-for-contributing-to-cleanlab).\n\n**Have questions?**  Check out [our FAQ](https:\u002F\u002Fdocs.cleanlab.ai\u002Fstable\u002Ftutorials\u002Ffaq.html) and [Github Issues](https:\u002F\u002Fgithub.com\u002Fcleanlab\u002Fcleanlab\u002Fissues?q=is%3Aissue).\n","cleanlab 是一个专注于数据质量和机器学习的开源库，特别适用于处理带有噪声标签的真实世界数据。其核心功能包括自动检测并修复数据集中的问题（如异常值、重复项、标签错误等），通过利用现有模型来估计和解决这些问题，从而帮助训练更准确的模型。该工具支持多种类型的数据集，包括文本、音频、图像以及表格数据，并且能够进行主动学习以建议优先标注的数据点。它非常适合需要提升数据质量以优化机器学习性能的各种应用场景，例如在构建高质量训练集时或是在多标注者环境下确保标签一致性。",2,"2026-06-11 03:24:00","top_topic"]