How to Use Cleanlab: Find Label Errors in Your ML Dataset (The Beginner's Guide)
CodexFuse Team · 2026-11-06T23:02:38.026Z · 17 min read min read
How to Use Cleanlab?
Learning how to use cleanlab takes four moves: install the library, get out-of-sample predictions from any classifier, audit the dataset with Datalab, and retrain on the cleaned data. Cleanlab flags the examples where the label and the model disagree, scores every label from 0 to 1, and lets you fix or drop the bad ones before you retrain. This cleanlab tutorial follows the official quickstart for version 2.9.0, the current release as of November 2026.
You will run every example in a Python notebook, so start with the Jupyter beginner guide on CodexFuse if you have never opened one. The guide below uses scikit-learn for its examples, because any classifier that outputs probabilities works with Cleanlab. No account, license key, or server is needed at any point.
- Install the library with pip.
- Train any classifier and collect out-of-sample predicted probabilities.
- Build a Datalab audit from your data and labels.
- Read the report and open the label issues it found.
- Check the label quality score of each flagged example.
- Fix or drop the bad labels.
- Retrain with CleanLearning on the cleaned data.
What Cleanlab Does (and What It Does Not Do)
Cleanlab is a free, open-source Python library that finds label errors and other data-quality issues in machine learning datasets. It compares each example's given label against what a trained model predicts for that example, using predicted class probabilities. To detect mislabeled data python developers pass their labels and their model's probabilities to Cleanlab, and Cleanlab returns which examples look wrong.
Cleanlab audits labels, it never creates them, so it is not a labeling tool. Argilla labels data and Cleanlab audits it, and the Argilla beginner guide on CodexFuse shows the labeling side of the same pipeline. CodexFuse offers no Cleanlab tool, so this guide teaches the library itself.
The library is built on confident learning, a method from MIT research that estimates the joint distribution of noisy and true labels. It works with any classifier because it operates on model outputs, not on the model itself, so PyTorch, TensorFlow, Hugging Face, XGBoost, and scikit-learn all work. The table below draws the line between Cleanlab and the three tools beginners confuse it with.
| Tool | Job | How it differs from Cleanlab |
|---|---|---|
| Cleanlab | Audits existing labels and finds errors | The auditor in this guide |
| Argilla | Humans label and correct data in a web UI | Creates and corrects labels, never audits them on its own |
| Distilabel | Generates synthetic data and AI feedback with LLMs | Creates new training data, never audits existing labels |
| Snorkel | Creates labels from scratch with labeling functions | Weak supervision for unlabeled data, not error-finding |
Install Cleanlab
Install Cleanlab with pip, and add the all extra if you want every optional dependency in one move. The library needs Python 3.10 or newer, according to the live PyPI metadata for version 2.9.0. The install takes under a minute on a normal connection, and there is no account, license key, or server to set up.
pip install cleanlab pip install "cleanlab[all]" uv pip install cleanlab
Before you run the examples, gather the five things in the table below. Each one is ordinary, and most beginners already have the first three.
| Requirement | Details |
|---|---|
| Python | Version 3.10 or newer |
| Installer | pip or uv |
| Example classifier | scikit-learn, any model that outputs probabilities |
| Labeled dataset | Labels as integers from 0 to K minus 1, for K classes |
| Model outputs | Predicted probabilities, one row per example, rows summing to 1 |
One trap is the old PyPI project page, which still says the package runs on Python 3.8 and newer, and the old Python 3.8 line on stale PyPI pages is wrong. Trust the live PyPI metadata, which says Python 3.10 or newer, and check the version you installed with pip show cleanlab.
Step 1: Get Out-of-Sample Predicted Probabilities
Cleanlab needs predicted probabilities that come from data the model never trained on, and these are called out-of-sample probabilities. A model that scores its own training data is overconfident, so its probabilities say more about memorization than about label quality. In-sample probabilities mislead, and every Cleanlab result downstream depends on getting this step right.
You can get these probabilities from any classifier, from a small scikit-learn model to a local large language model. The Ollama beginner guide on CodexFuse shows how to run a model locally when you want embeddings or probabilities without a cloud service. For the examples in this guide, scikit-learn is enough, and cross-validation does the out-of-sample work for you.
from sklearn.linear_model import LogisticRegression from sklearn.model_selection import cross_val_predict model = LogisticRegression(max_iter=1000) pred_probs = cross_val_predict(model, X, labels, cv=5, method="predict_proba")
The result is a table with one row per example and one column per class, and every row sums to 1. Cross-validation trains five copies of the model on different folds, so each example is scored by a copy that never saw it. That is the whole trick, and it is also why Cleanlab runs slower on big models: the cost is five training runs, not one.
Step 2: Audit Your Dataset with Datalab
Datalab is the fastest way to use Cleanlab, because one audit checks seven issue types at once. This cleanlab datalab tutorial uses the Datalab class with your dataset, the name of the label column, your feature embeddings, and your predicted probabilities. The audit returns a report that ranks every issue by severity, so you start with the worst problems first.
from cleanlab import Datalab lab = Datalab(data=your_dataset, label_name="column_name_of_labels") lab.find_issues(features=feature_embeddings, pred_probs=pred_probs) lab.report()
The report lists each issue type, how many examples it affects, and an overall quality score for the dataset. Datalab checks label errors, outliers, near duplicates, non-IID ordering, class imbalance, underperforming groups, and missing values by default. A data valuation check is available too, but you must opt in to it.
| Issue type | What it finds | Needs |
|---|---|---|
| label | Examples whose given label looks wrong | Predicted probabilities or features |
| outlier | Examples far from the rest of the data | Predicted probabilities, features, or neighbor graph |
| near_duplicate | Near-identical examples with possibly different labels | Features or neighbor graph |
| non_iid | Ordering patterns that break the random-sample assumption | Predicted probabilities, features, or neighbor graph |
| class_imbalance | Classes with far fewer examples than the rest | Nothing extra |
| underperforming_group | Groups of examples the model handles badly | Predicted probabilities plus features |
| null | Missing values in the features | Features |
| data_valuation | Examples that hurt or help the model most | Neighbor graph, opt-in only |
Open the label issues first, because a wrong label teaches the model the wrong lesson on every epoch. Outliers and near duplicates matter too, but they rarely damage a model as fast as a mislabeled example. The next section shows the focused tool for label errors alone.
Step 3: Find Label Errors with find_label_issues
When you only care about cleanlab label errors, the find_label_issues function is the direct route, with no Datalab object needed. To find label errors in dataset python code, you pass two arrays: labels, a one-dimensional array of integers from 0 to K minus 1, and pred_probs, a two-dimensional array with one row per example and one column per class. The function returns a boolean mask by default, where True marks an example Cleanlab believes is mislabeled.
The cleanlab find_label_issues example below shows the call in its simplest form, with the default filter that prunes by the estimated noise rate. If the word API is new to you, the what an API is guide on CodexFuse explains the idea in plain terms. The function signature also accepts a ranking mode, which returns the flagged indices sorted from most likely error to least.
from cleanlab.filter import find_label_issues issues = find_label_issues(labels, pred_probs) print(issues.sum(), "examples flagged out of", len(labels))
Start with the defaults, and only change filter_by when you understand what each option keeps and drops. The mask is the honest output: it tells you which rows to inspect, and a human still makes the final call. Cleanlab never rewrites your labels on its own, and that is by design.
| Argument | What it does |
|---|---|
| filter_by | Default prune_by_noise_rate, with prune_by_class, both, confident_learning, predicted_neq_given, low_normalized_margin, and low_self_confidence as options |
| return_indices_ranked_by | None returns a boolean mask, while self_confidence, normalized_margin, or confidence_weighted_entropy return ranked indices |
| frac_noise | Default 1.0 returns every flagged example, and lower values return only the top fraction |
| n_jobs | Defaults to the CPU core count, and 1 disables multiprocessing when errors appear |
| multi_label | True for multi-label data such as image or document tagging |
Step 4: Read the Label Quality Score
Every example gets a label quality score, and the cleanlab label quality score is a number from 0 to 1 that says how likely the given label is correct. A score of 1 means the label looks clean, and a score of 0 means it looks dirty, so low scores deserve your attention first. The docs show one example with a score of 0.064, a given label of 0, and a predicted label of 2, which is a 6.4 percent confidence in the given label.
from cleanlab.rank import get_label_quality_scores scores = get_label_quality_scores(labels, pred_probs) print(scores[:5])
Sort your data by score and inspect the lowest rows first, because that is where the real errors hide. A score near 0.5 means the model is unsure, which can mean a hard example rather than a wrong label, so context matters. Scores only mean something when they come from out-of-sample probabilities, so never score in-sample predictions.
| Method | How the score is computed | Best for |
|---|---|---|
| self_confidence | The predicted probability of the given label | Default choice, simple and fast |
| normalized_margin | Probability of the given label minus the best other label | Class-conditional label errors |
| confidence_weighted_entropy | Confidence weighted by the spread of the probabilities | Noisy predictions across many classes |
There is no universal cutoff, because a 0.2 in a clean dataset means something different from a 0.2 in a noisy one. Look at the distribution, inspect the bottom rows by hand, and fix what is plainly wrong. The score is a ranking tool, not a verdict.
Step 5: Train a Robust Model with CleanLearning
CleanLearning wraps any scikit-learn-compatible classifier and trains a robust version of it on your noisy labels. You hand it an untrained model, your data, and your labels, and it cross-validates to get out-of-sample predictions, finds the label issues, and fits the final model on the cleaned data. The result is a model trained only on the clean rows, and you call predict on it exactly like the model you wrapped.
from sklearn.linear_model import LogisticRegression from cleanlab.classification import CleanLearning cl = CleanLearning(clf=LogisticRegression()) cl.fit(train_data, labels) predictions = cl.predict(test_data)
The same wrapper works with PyTorch models through the skorch package, and with Hugging Face models that expose probabilities. The LangChain beginner guide on CodexFuse shows the application side, where a cleaned dataset makes a retrieval pipeline noticeably more reliable. CleanLearning also exposes the issues it found, so you can inspect them with cl.get_label_issues() before you trust the model.
Expect training to take several times longer than a single fit, because cross-validation trains one copy of your model per fold. That cost buys you a model that never saw the mislabeled rows during its final fit. For most beginners, that trade is worth it.
Cleanlab for Text, Images, and Regression
Cleanlab works on text, image, tabular, and audio datasets, because it only needs model outputs, not a particular data type. The Hugging Face cookbook audits 1,000 customer-service requests from the Banking77-OOS dataset with an Electra embedding model and logistic regression. The audit found 62 issues in those 1,000 examples, which is 6.2 percent of the data, including 10 label errors, or 1.0 percent.
| Data type | Worked example | Issues found |
|---|---|---|
| Text | Banking77-OOS, 1,000 customer-service requests | 62 issues, 6.2 percent, including 10 label errors |
| Image | Fashion-MNIST, 60,000 images | 7,714 issues, 12.86 percent of the set |
| Token classification | CoNLL-2003, about 300,000 tokens | 2,254 potential label issues, 0.75 percent |
| Regression | Student grades, numeric labels | More than 100 flagged rows in the tutorial |
The image tutorial audits 60,000 Fashion-MNIST images and finds 7,714 issues, which is 12.86 percent of the set. The token classification tutorial audits about 300,000 CoNLL-2003 tokens and finds 2,254 potential label issues, which is 0.75 percent. The regression tutorial works on student grades, where Cleanlab fits several copies of a regression model and flags the rows whose numeric labels look corrupted, reporting more than 100 flagged rows.
from cleanlab.regression.rank import get_label_quality_scores label_quality_scores = get_label_quality_scores(y, predictions)
import datetime
today = datetime.date(2026, 11, 7)
release = datetime.date(2026, 1, 13)
print("days since v2.9.0:", (today - release).days)
print("HF cookbook issue rate:", round(62 / 1000 * 100, 1))
print("Fashion-MNIST issue rate:", round(7714 / 60000 * 100, 2))
print("CoNLL-2003 issue rate:", round(2254 / 300000 * 100, 2))
print("stars per fork:", round(11735 / 940, 2))
# outputs: 298, 6.2, 12.86, 0.75, 12.48
These are worked tutorial outputs, not guarantees for your data, and your own issue rate will depend on how your labels were made. Crowdsourced labels tend to carry more errors than expert labels, and scraped labels carry the most of all. Run the audit before you blame the model, because the data is usually the problem.
Cleanlab vs Cleanlab Studio
The open-source library is free under the Apache-2.0 license, and it runs on your own machine with your own model. Cleanlab Studio is the hosted product: a no-code web platform that runs the same algorithms on AutoML and foundation models, with a correction interface and AI-suggested fixes. Studio offers a 2-week free trial and paid tiers, and the library stays free either way.
The company homepage now leads with GenAI agent guardrails under the Codex name, which is a different product from the library in this guide. That shift does not change the library: version 2.9.0, released on 2026-01-13, is the current stable release, 298 days before this guide. The library is a mature, stable tool, not a fast-moving one, and the examples in this guide use its current API.
Prices change, so check the official pricing page before you budget for Studio, and never trust a third-party price list. This guide quotes no dollar figures for Studio, because the official pricing page fetched for this guide carried no current prices. The library itself costs nothing, and that is the only price in this article.
Check Overall Dataset Health in One Line
After you fix the label errors, check the whole dataset with health_summary, which scores every class and the dataset as a whole. The function takes your labels, your predicted probabilities, and your class names, and it reports overlapping classes worth merging and classes worth keeping or deleting. Run it before you clean and after, and the score shows whether your work helped.
from cleanlab.dataset import health_summary health_summary(labels, pred_probs, class_names=class_names)
Overlapping classes are a common find, and they mean two labels describe nearly the same thing, so the model cannot tell them apart. Merging them is often a better fix than relabeling hundreds of examples. Track the health score across cleaning rounds, and stop when it stops improving.
The code examples in this guide come from the official cleanlab docs for version 2.9.0, and no testbed outputs were executed for this article. Run each snippet on a small slice of your own data first, and inspect the flagged rows before you change anything. Cleanlab points at problems, and you decide what to do with them.
Seven Mistakes Beginners Make with Cleanlab
Most Cleanlab failures come from setup, not from the library, and the seven mistakes below cover nearly all of them. Read the table once before your first audit, and you will avoid the errors that waste the most time. Each mistake has a one-line fix, and the fix is usually a single argument or a different function call.
| Mistake | Fix |
|---|---|
| Scoring in-sample probabilities | Use cross-validated out-of-sample probabilities |
| Copying v1 syntax such as LearningWithNoisyLabels or get_noise_indices | Use CleanLearning and find_label_issues |
| Passing label to Datalab | Pass label_name= |
| Trusting the old Python 3.8 line | Install on Python 3.10 or newer |
| Quoting the old under 1 second on ImageNet line as current | No current benchmark exists, so quote none |
| Expecting Cleanlab to rewrite labels | Inspect flagged rows and fix them yourself |
| Treating a low score as proof of an error | Inspect the row, because a hard example can score low too |
Sources
The version, star, fork, and issue counts in this guide come from the GitHub API for cleanlab/cleanlab and the PyPI JSON API, both fetched on 2026-11-07. The release dates come from the GitHub releases API, which lists version 2.9.0 as the latest release, published on 2026-01-13. The day counts and percentages were recomputed in Python for this guide, and the repo itself was created on 2018-05-11, which is 3,102 days before 2026-11-07.
The code examples and function signatures come from the official cleanlab docs quickstart for version 2.9.0, the cleanlab.filter reference, the Datalab issue-type guide, the cleanlab.rank reference, the regression tutorial, and the in-depth workflows tutorial, all fetched on 2026-11-07. The product and pricing notes come from the cleanlab.ai homepage and pricing page, with the pricing page carrying a stale crawl from 2025-07-18, so no dollar figures are quoted. The Analytics Vidhya article on data preprocessing with Cleanlab, published on 2025-04-23, is the only independent walkthrough found on page 1 of the tutorial search.
The worked numbers come from the Hugging Face cookbook on detecting issues in a text dataset, which audits 1,000 Banking77-OOS requests and finds 62 issues including 10 label errors, and from the official tutorials for Fashion-MNIST image classification and CoNLL-2003 token classification. The Argilla comparison comes from the Argilla v1 tutorial on monitoring text classification with Cleanlab, the Distilabel comparison comes from the Argilla blog introduction to Distilabel and the Distilabel PyPI page, and the Snorkel framing comes from a third-party cleanlab profile. Two live search queries for how to use cleanlab and cleanlab tutorial were run on 2026-11-07, and the GitHub and PyPI numbers were re-checked directly by the writer this run.
Frequently Asked Questions
Is Cleanlab free?
Yes. The cleanlab library is free and open source under the Apache-2.0 license, and you can install it with pip and run it on your own machine. Cleanlab Studio, the hosted no-code product, offers a 2-week free trial and paid tiers, with current prices on the company's pricing page.
Does Cleanlab need a GPU?
No. Cleanlab itself runs on CPU and parallelizes across all CPU threads by default. You only need a GPU when your own model or embedding step needs one, for example a transformer that builds the embeddings.
What is a good label quality score?
Scores run from 0 to 1, and higher means the label looks more likely correct. There is no universal cutoff, so sort your data by score and inspect the lowest rows first. A score near 1 means clean, a score near 0 means likely wrong, and a score near 0.5 usually means a hard example rather than a wrong label.
Can Cleanlab fix my labels automatically?
No, not in the open-source library. Cleanlab flags the likely errors and suggests a predicted label for each one, but you decide what to change. Cleanlab Studio adds AI-suggested fixes in a correction interface, and the library itself never rewrites your data.
Does Cleanlab work with PyTorch and Hugging Face?
Yes. Cleanlab works with any classifier that outputs probabilities, including PyTorch, TensorFlow, Hugging Face, XGBoost, and scikit-learn. PyTorch models need a scikit-learn-compatible wrapper, and the docs point to the skorch package for that job.
