How to Use Argilla: Label Your First AI Dataset (From Zero to Human-in-the-Loop)

CodexFuse Team · 2026-11-06T22:11:35.468Z · 16 min read min read

How to Use Argilla to Label Your First AI Dataset?

To use Argilla, you deploy its server, connect the Python SDK with an API key, define a dataset with fields and questions, load records, and label them in the web UI. Argilla is an open-source platform for building the labeled datasets that AI models learn from, and this guide walks the whole arc from zero to a labeled export. Everything here is current as of November 2026 and uses the v2 API, which replaced the older v1 API.

The short version is seven steps, and the rest of this guide expands each one. You will run the Python SDK from a notebook, so work through the Jupyter beginner guide first if notebooks are new to you.

1. Deploy Argilla on Hugging Face Spaces or with Docker Compose.
2. Install the Python SDK with pip install argilla.
3. Connect a client with your server URL and API key.
4. Define dataset settings with fields and questions.
5. Create the dataset and log your records.
6. Label the records in the Argilla UI.
7. Export the labeled dataset to the Hub or to disk.

Argilla is the labeling UI plus the data layer your annotators click through, not the training loop itself. Your model trains elsewhere, and Argilla hands it clean labeled data.

What Is Argilla, Exactly?

Argilla is a free, open-source, self-hosted platform for data-centric LLM development, and its recommended deploy route runs on Hugging Face Spaces. It gives teams a web UI where humans review records and answer questions about them, plus a Python SDK that moves datasets in and out. The name also belongs to a clay plaster brand, so search results mix wall-coating guides with the software, and this article is only about the software.

Argilla fits the feedback step of AI work: traditional NLP tasks such as text classification, LLM work such as RAG and preference tuning, and evaluation of model outputs. It is the place where human judgment becomes training data, and it stays out of the training itself.

This argilla tutorial sticks to the current v2 API from the live docs and the live repository. CodexFuse offers no Argilla tool, so this guide teaches the real project, not a CodexFuse page.

If the model you are labeling runs on your own machine, the Ollama beginner guide shows the local-model setup. The project lives at argilla-io/argilla on GitHub, where it holds 5,193 stars and 514 forks with 39 open issues, and it was created on 2021-04-28.

It is 2,019 days old, which is about 5.53 years, and it carries the Apache-2.0 license. That works out to 10.1 stars per fork, a 9.9 percent fork ratio, 0.75 open issues per 100 stars, and 2.57 stars per day since creation.

from datetime import date
today = date(2026, 11, 7)
created = date(2021, 4, 28)
latest = date(2025, 3, 11)
previous = date(2025, 2, 6)
pushed = date(2026, 11, 2)
stars, forks, issues = 5193, 514, 39
age = (today - created).days
print(age, round(age / 365.25, 2), (today - latest).days, (latest - previous).days, (today - pushed).days, round(stars / forks, 2), round(forks / stars * 100, 2), round(issues / stars * 100, 2), round(stars / age, 2), round(28643 / stars, 2), round(stars / 28643 * 100, 1))
# Output: 2019 5.53 606 33 5 10.1 9.9 0.75 2.57 5.52 18.1

What Do You Need Before You Start?

You need a Hugging Face account if you take the Spaces route, or Docker on your machine if you take the local route. You also need Python 3.9 or newer on the machine that runs the SDK, plus a small sample of the text you want to label.

Most teams label data for a downstream job, such as a classifier or a retrieval pipeline, so know which job yours is before you design questions. Teams that feed labeled data into a retrieval pipeline usually build it with LangChain, so the LangChain beginner guide is the natural next read.

Argilla Server Setup: Hugging Face Spaces or Docker Compose

Argilla is a self-hosted tool, which means you deploy its UI before you use it, and the docs offer two routes. Hugging Face Spaces is the recommended start because it needs no server of your own, while Docker Compose runs the whole stack on your machine or a server you control.

If Docker itself is new, the Docker beginner guide covers the install first. On Spaces, set persistent storage to SMALL or you will lose your data when the Space restarts, because Spaces restart for maintenance, inactivity, and settings changes.

pip install argilla

mkdir argilla
cd argilla
wget -O docker-compose.yaml https://raw.githubusercontent.com/argilla-io/argilla/main/examples/deployments/docker/docker-compose.yaml
docker compose up -d
docker compose logs -f

import argilla as rg
client = rg.Argilla.deploy_on_spaces(api_key="owner.apikey")

Spaces signs you in with Hugging Face OAuth, and you arrive as the owner of your Space. Docker signs you in with the username and password from the compose file, which you should change before any real use.

The compose file ships with default credentials, so treat the out-of-the-box login as a local demo login and replace it before the server touches a network.

Your API key lives on the My Settings page of your Argilla Space, and the client needs it plus the Space URL to connect. If that term is new, read what an API is before you copy the key.

If the Docker deploy fails, check Elasticsearch first, because the deploy guide says most deployment issues are related to it. The UI appears at localhost:6900 once the stack is up, and the server and the UI arrive together in the same image.

How Do You Create Your First Argilla Dataset?

A dataset in Argilla is defined by Settings: guidelines in plain language, fields that hold the data to judge, and questions that collect the feedback. You write the settings in Python, create the dataset, and it appears in the UI ready for annotation.

import argilla as rg
client = rg.Argilla(
    api_url="https://your-owner-name-your-space-name.hf.space",
    api_key="owner.apikey"
)

settings = rg.Settings(
    guidelines="Classify the reviews as positive or negative.",
    fields=[
        rg.TextField(
            name="review",
            title="Text from the review",
            use_markdown=False,
        ),
    ],
    questions=[
        rg.LabelQuestion(
            name="my_label",
            title="In which category does this article fit?",
            labels=["positive", "negative"],
        )
    ],
)
dataset = rg.Dataset(
    name="my_first_dataset",
    settings=settings,
    client=client,
)
dataset.create()

Call client.me to confirm the connection returns your user information before you create anything. The fastest first dataset skips code entirely: open the Argilla home page, choose Import dataset from Hugging Face, and pick a sample dataset.

Argilla maps the columns to fields and questions for you, and it imports only the first 10,000 rows, so larger sets still need the SDK. Dataset settings can be edited after creation, so a wrong question title is fixable without starting over.

Workspaces group datasets and user access, and the workspace parameter is optional because the default workspace is called argilla. That default keeps the first dataset simple, and you can add workspaces later when a team joins.

How Do You Add Records to an Argilla Dataset?

Records are the rows your annotators judge, and each one carries fields, metadata, and optional vectors or model suggestions. You add them from Python with dataset.records.log, passing your rows and a mapping from your column names to the dataset fields.

pip install datasets
from datasets import load_dataset
data = load_dataset("imdb", split="train[:100]").to_list()
dataset.records.log(records=data, mapping={"text": "review"})

That example loads the first 100 rows of the IMDb training split and maps its text column onto the review field. Once the log call finishes, the records are visible in the UI and ready to annotate.

A record can also carry a suggestion, which is a model-proposed label a human reviews instead of writing from scratch. Suggestions speed up labeling, but a human still confirms every answer before it counts.

How Does Argilla Annotation Work in the UI?

Annotation in Argilla means working through records one by one and answering the questions your dataset defines. Each record shows its fields, and you pick a label, give a rating, rank options, or type free text, depending on the question types you configured.

Filters and semantic search help you prioritize which records to label first, so you can start with the uncertain or unusual ones. The UI is a Vue.js web app that ships inside the same server image, so there is no separate UI container to deploy.

Labeling goes faster when the guidelines are short and the question list is small, so keep your first dataset to one field and one question. Add a second question only after the first labeling pass feels smooth.

How Does Argilla Active Learning Work?

Active learning in Argilla is the human-in-the-loop loop: a model suggests labels, a human reviews and corrects them, and the corrected labels feed the next round of training or ranking. The loop keeps humans on the decisions that matter and lets the model handle the easy cases.

Never copy v1 syntax into a v2 project, because the old listener code from the v1 active-learning tutorial will not run under v2. There is also no v2-native active-learning how-to in the current docs, so treat the loop above as the concept and follow the v2 how-to guides for the exact calls.

similar = dataset.find_similar_records(vector_name="my_vector", value=[0.1, 0.2, 0.3])

Semantic search runs on the search backend, which is Elasticsearch or AWS OpenSearch, and it finds records similar to an embedding you supply. Use it to pull near-duplicates together or to find the records a model is least sure about.

How Do You Export a Labeled Dataset from Argilla?

Export turns the labeled records back into a dataset your training code can read, and Argilla offers three routes. The Hub route publishes settings plus records, the disk route saves a local copy, and the records route hands you plain Python objects.

dataset.to_hub(repo_id="my_org/my_dataset")
dataset.to_disk(path="my_dataset")
records = dataset.records.to_list(flatten=True)

There is no CSV export button in the documented export flow, so plan on the Hub, disk, or Python routes rather than hunting for one. The datasets-object route is the recommended path for image data, where images arrive as rescaled PIL objects.

Argilla vs Label Studio: Which One Should You Pick?

Label Studio is the older generalist, built by HumanSignal, and it labels text, image, audio, video, and time-series data with custom interfaces. Argilla is the Hugging Face-native specialist, and its workflows target LLM feedback loops such as supervised fine-tuning and preference tuning.

Label Studio holds 5.52 times as many stars as Argilla, which makes Argilla 18.1 percent of the Label Studio star count. Pick Label Studio for breadth across data types and pick Argilla when your data lives on the Hugging Face Hub and your job is LLM feedback.

Both tools are open source under Apache-2.0, so the choice is about fit, not price. A team that labels text for fine-tuning will move faster in Argilla, while a team that labels images, audio, and video will find more in Label Studio.

Argilla, Cleanlab, and Distilabel: Three Different Jobs

Three tools from the same neighborhood do three different jobs, and mixing them up wastes a week. Argilla labels data, Cleanlab audits labels, and Distilabel generates synthetic data, so they complement each other instead of competing.

Cleanlab holds 11,735 stars and finds label errors in datasets you already have, but it runs no annotation UI. Distilabel holds 3,452 stars and builds synthetic datasets from research-backed pipelines, and its output is exactly the kind of data Argilla is built to curate.

A practical pipeline is Distilabel first for synthetic rows, Argilla next for human review, and Cleanlab last for an audit of the finished labels. Each tool keeps its own lane, and the labeled dataset that comes out is stronger for it.

Seven Mistakes Beginners Make With Argilla

Most first-time Argilla setups fail in the same seven ways, and all seven are avoidable. Read the table once before you deploy, and your first dataset will take an afternoon instead of a week.

One more honest note: the latest tagged release is v2.8.0 from 2025-03-11, which is 606 days old, yet the last commit landed 5 days ago. The project is not abandoned and not freshly released, so pin your install to v2.8.0 and read the current docs instead of assuming the newest code. Version 2.8.0 landed 33 days after v2.7.1, which was published on 2025-02-06.

Do not quote a price for Argilla Cloud, because the managed tier has no publicly listed price and its current status is unclear across 2026 sources. The current docs describe Argilla only as a free, open-source, self-hosted tool with the Spaces and Docker routes.

One last trap: the README link to docs.v2.argilla.io returns a 404, so use docs.argilla.io for the current documentation. The v1 docs still live at docs.v1.argilla.io, and they are useful history, not current instructions.

Sources

The project facts in this guide come from the GitHub API record for argilla-io/argilla, the GitHub Releases API, the raw README on the main branch, and the PyPI JSON API for the argilla package. The deploy steps, the Settings model, and the export calls come from the current quickstart, the Docker deploy guide, the raw Docker Compose file, and the import and export guide on docs.argilla.io. The search picture comes from live searches for how to use argilla and argilla tutorial, plus a search for an argilla versus Label Studio comparison.

The comparison numbers come from the GitHub API records for HumanSignal/label-studio, argilla-io/distilabel, and cleanlab/cleanlab. The third-party context comes from the 7wData Argilla profile and the potatoannotator self-hosted comparison from October 2026, and the v1 history comes from the v1.25 cheatsheet and the PyPI page for argilla 1.19.0. Every number in this article was recomputed from the fetched values with a short Python script run during writing.

Two facts could not be verified and are not asserted as facts here: the exact record-status names in the v2 UI and a v2-native active-learning how-to in the current docs. The managed-tier pricing could not be verified either, so no price appears in this guide. Star and fork counts are a 2026-11-07 snapshot, and no testbed outputs were executed while writing.

Frequently Asked Questions

Is Argilla free to use?

Yes, Argilla is free and open source under the Apache-2.0 license, and the current docs describe it as a free, self-hosted tool. You pay only for the machine or Space that runs it, and there is no paid feature gate in the open-source edition.

What Python version does Argilla need?

Argilla needs Python 3.9 or newer, according to the requires_python field on PyPI. Install the SDK with pip install argilla in that Python, and keep the SDK on the same machine that runs your notebooks.

Can I use Argilla without Docker?

Yes, deploy on Hugging Face Spaces with one click and no commands, then install only the Python SDK locally. Docker is only needed when you want the server on your own machine.

How is Argilla different from Label Studio?

Argilla is the Hugging Face-native specialist for LLM feedback loops, while Label Studio is the broader generalist across text, image, audio, and video. Both are open source under Apache-2.0, and the comparison table above lays out the numbers.

How do I export my labeled data from Argilla?

Use dataset.to_hub to publish to the Hugging Face Hub, dataset.to_disk for a local copy, or dataset.records.to_list for plain Python objects. There is no documented CSV export button, so use one of those three routes.