How to Use Great Expectations: The Beginner's Guide

CodexFuse Team · 2026-11-03T12:20:38.936Z · 14 min read min read

A beginner's guide to Great Expectations: install it, create a Data Context, write your first Expectation Suite as a data contract, run a Checkpoint, and read Data Docs — using only the current 1.0+ API.

If your pipelines silently ship bad data — nulls where customer names should be, duplicate orders, dates from the future — you need a quality layer that says "no" before the damage spreads. This Great Expectations tutorial shows you how to use Great Expectations as a complete beginner: install it with pip, create a Data Context, write your first Expectation Suite, point it at real data, and run your first Checkpoint so Data Docs prove your data keeps its promises. Written for data engineers in November 2026, it uses only the current 1.0+ API — none of the dead 0.x commands the old tutorials still teach. (And no, not the Dickens novel — this is the open-source Python data-quality framework.) If you are the one building the tables GX will check, start with the dbt beginner guide — GX is the validation layer that sits around your transforms, not a replacement for them.

What Is Great Expectations, Exactly?

Great Expectations (GX) is an open-source Python framework for data-quality testing. You describe what "good data" looks like as Expectations — individual assertions such as "this column never contains nulls" or "this table holds between 1,000 and 1,000,000 rows" — group them into an Expectation Suite, and validate every batch of data against that suite before it moves downstream.

The core mental model is the suite-as-contract. As GX co-creator Abe Gong puts it, the best framing for data contracts is to think of them as a set of expectations — and the test blog nadeblg.com adds the sharp edge: if a contract fails, the pipeline should stop before bad data propagates, and that is what makes it a contract and not just a test. GX ships more than 300 built-in Expectations (per the conduktor.io GX glossary), so you rarely write a check from scratch. The core is Apache 2.0 and, in the vendor's own words, "will always remain free" (greatexpectations.io).

What You Need Before You Start

You need three things: Python with a virtual environment, a dataset worth checking, and about an hour. The official quickstart says to run the install "in an empty base directory inside a Python virtual environment" — take that literally, because GX's dependency tree is heavy and you do not want it fighting your project packages.

Two honest caveats before you begin. First, GX's docs disagree with each other about the exact Python version range (the quickstart says one range, a newer tutorial says another), so check the current supported-versions page instead of trusting any single number here. Second, the official quickstart currently lists Windows support for GX OSS as unavailable — if you are on Windows, plan on WSL or a Linux VM.

How Do I Install Great Expectations?

Install is one command, run inside your virtual environment:

python -m venv gx-env
source gx-env/bin/activate   # gx-env\Scripts\activate on WSL
pip install great_expectations

That wording — "Run the following command in an empty base directory inside a Python virtual environment: pip install great_expectations" — is the official quickstart's own instruction, and it has not changed in the 1.x line. Ignore any tutorial that opens with great_expectations init or great_expectations suite new: that CLI workflow died with GX 1.0 in August 2024, and following it will leave you with commands your installed version no longer understands. The rest of this guide never touches the old CLI.

What Does the First-Run Arc Look Like?

Your first run has five steps, and the order is the lesson:

1. Install — pip install great_expectations inside a fresh virtual environment.
2. Get the context — gx.get_context() gives you a Data Context, the object everything else hangs off.
3. Write the suite — context.suites.add(...) creates your Expectation Suite, and suite.add_expectation(...) adds each rule.
4. Wire the data — add a data source, a data asset, and a batch definition, then bind the suite to the batch with a validation definition.
5. Run a Checkpoint — checkpoint.run() executes everything and produces Data Docs you can read.

Notice what is missing: no CLI scaffolding, no YAML editing, no Validator object. In the 1.0+ fluent API every step is Python, and every step goes through the context you created in step two.

What Is the Data Context, and Why Does It Come First?

The Data Context is the front door to everything in GX: data sources, suites, validation definitions, and checkpoints all live on it. The 1.0+ way to get one is a single call:

import great_expectations as gx

context = gx.get_context()

This creates an ephemeral, in-memory context — nothing is saved to disk, which makes it perfect for learning and for scripts. GX also supports project contexts that persist configuration, but start ephemeral: you can rebuild it in one line, and you cannot corrupt anything you will miss. Every object you create for the rest of this guide comes from context, and that is the first rule of the 1.0+ mental model — nothing in GX 1.x floats free of the context.

How Do I Create My First Expectation Suite?

An Expectation Suite is the contract: a named list of rules your data must satisfy. Create it through the context and add one expectation at a time:

import great_expectations as gx

context = gx.get_context()

suite = context.suites.add(
    gx.core.expectation_suite.ExpectationSuite(name="customer_orders_suite")
)

suite.add_expectation(
    gx.expectations.ExpectTableRowCountToBeBetween(min_value=1000, max_value=1000000)
)
suite.add_expectation(
    gx.expectations.ExpectColumnValuesToNotBeNull(column="customer_id")
)

The class names look verbose, but the pattern never changes: gx.expectations.Expect<What><Condition>. To plan a suite for a real table, sketch the columns and the rule each one needs — here is a computed example for a customer_orders.csv:

Write the suite before you look at the data's problems. That ordering is what turns a checklist into a contract: you are declaring what the data must be, not documenting what it happens to be.

How Do I Point Great Expectations at Real Data?

GX 1.0+ connects data in three layers, each created through the context. For a local CSV or DataFrame the chain is:

data_source = context.data_sources.add_pandas("pandas_datasource")
data_asset = data_source.add_dataframe_asset(name="customer_data")
batch_definition = data_asset.add_batch_definition_whole_dataframe("customer_batch")

The three layers each do one job. The data source knows how to reach a system (here, Pandas). The data asset names one dataset inside that system (here, your DataFrame). The batch definition picks which slice to validate — "the whole DataFrame" is the beginner's choice; production setups define batches per file, per partition, or per day. Batch Definitions are the 1.0+ replacement for the old batch_request pattern, and if you are copying code from a pre-2024 tutorial, this is the first place it will break.

Real projects rarely validate CSVs — they validate tables in a warehouse. The Pandas path you just learned transfers directly to SQL sources, and if you want a warehouse that runs in a single file, the DuckDB beginner guide is the fastest way to get one.

How Do I Run a Validation and Read the Results?

Two more objects connect your batch to your suite, then one call runs everything:

validation_definition = context.validation_definitions.add(
    gx.core.validation_definition.ValidationDefinition(
        name="customer_validation",
        data=batch_definition,
        suite=suite,
    )
)

checkpoint = context.checkpoints.add(
    gx.checkpoint.checkpoint.Checkpoint(
        name="customer_checkpoint",
        validation_definitions=[validation_definition],
    )
)

checkpoint_result = checkpoint.run()
print(checkpoint_result.success)

A validation definition binds one batch to one suite — "validate this data against that contract." A Checkpoint is the runnable unit: it can hold many validation definitions and is what schedulers call. checkpoint.run() returns a result with a simple boolean, checkpoint_result.success, and — more useful — it generates Data Docs: auto-built HTML reports showing every expectation, which rows failed, and when the run happened. The conduktor.io glossary calls Data Docs "living documentation of your data contracts," and that is exactly how to read them: not test logs, but the current state of your promises to downstream consumers.

Great Expectations vs Soda: Which Should I Pick?

Great Expectations and Soda are the two open-source names that come up in every data-quality conversation, and they approach the problem from opposite ends. The data-quality blog dataexpert.io (Feb 2026) draws the line cleanly: "Soda is SQL/YAML-based, easy to set up, and great for real-time monitoring. Great Expectations is Python-based, flexible, and ideal for complex validations and documentation." The monitoring directory branchboston.com adds that "Soda (Soda Core / Soda Cloud) — a lightweight scanner and observability tool that can run checks defined in YAML (SodaCL) and offers templated checks, monitoring, and alerting."

Both have free open-source versions — Soda Core and GX OSS — so trying each costs nothing but an afternoon. The honest answer for most teams is not either/or: Soda-style YAML checks catch drift in production tables while a GX suite enforces the contract at the pipeline gate.

Five Beginner Mistakes to Avoid

Almost every GX beginner trips on one of these, and four of the five trace back to trusting outdated material:

How Does Great Expectations Fit a Real Pipeline?

A Checkpoint is designed to be called, not clicked — which is why the suite-as-contract model only pays off when it sits inside an orchestrator. The typical shape: your transforms run, your GX Checkpoint validates the output, and only passing data moves to the warehouse and the BI layer. A failed contract pages someone instead of quietly poisoning dashboards.

If you run that pattern on a schedule, the orchestrator that calls your Checkpoints matters as much as the Checks themselves — the Kestra beginner guide covers one that fits this job well. And if you have not yet decided where the transforms live, revisit the dbt beginner guide — GX validates the data contract, dbt owns the transform logic, and keeping those two jobs in two tools is what makes both maintainable.

What Should I Learn Next?

Once the first arc runs end to end, three directions are worth your time. First, productionize the context: move from the ephemeral gx.get_context() to a project context with stored configuration, and schedule Checkpoints through your orchestrator instead of running them by hand. Second, study the 300+ built-in Expectations before writing custom ones — most "I need a custom check" moments turn out to be a parameter you missed. Third, if your team is YAML-first rather than Python-first, evaluate Soda Core alongside GX OSS; the comparison table above is your starting map. GX Cloud exists as a paid hosted option if you want managed Data Docs and scheduling, but its public FAQ publishes tier structure rather than dollar prices, so confirm current terms with the vendor before budgeting.

Sources

PyPI great-expectations release history (reviewed 2026-11-03) documents GX 1.24.1 released October 22, 2026; the conduktor.io Great Expectations glossary (crawled 2026-11-03) documents the GX 1.0 August 2024 breaking changes, the 300+ built-in Expectations, and Data Docs as "living documentation of your data contracts"; the official GX retirement blog (greatexpectations.io/blog/retiring-gx-0-18, fetched 2026-11-03) documents the October 1, 2025 sunset of 0.18 in GX Cloud and the removal of the older Custom Expectations implementation and Value-Based Batches; greatexpectations.io (fetched 2026-11-03) documents the Apache 2.0 license and that GX Core "will always remain free"; the GX Cloud FAQ (greatexpectations.io/gx-cloud-faqs, fetched 2026-11-03) documents the tier structure — developer plan up to 5 Data Assets and 3 users, unlimited Expectations and rows on paid plans, 99.5% SLA and SOC 2 Type II on Enterprise; the official quickstart (docs.greatexpectations.io, fetched 2026-11-03) documents the pip install wording, the empty-base-directory virtual-environment instruction, and that GX OSS Windows support is currently unavailable; the bixtech.ai production-pipelines guide and the ilirivezaj.com tutorial (both fetched 2026-11-03) corroborate the 1.0+ fluent API chain; the kdnuggets.com data-quality guide and the official greatexpectations.io MLOps article (fetched 2026-11-03) corroborate the pipeline-validation use case; dataexpert.io (Feb 13, 2026, fetched 2026-11-03) documents the Soda SQL/YAML vs GX Python framing and the free open-source versions of both; branchboston.com (fetched 2026-11-03) documents Soda's SodaCL YAML checks and alerting; the nadeblg.com data-contracts piece (fetched 2026-11-03) documents the "contract fails, pipeline stops" framing; uplatz.com (fetched 2026-11-03) documents "shared data contracts (Expectation Suites)".

Frequently Asked Questions

Is Great Expectations free to use?

Yes. GX Core is open source under Apache 2.0 and the vendor states it "will always remain free." There is a paid hosted product, GX Cloud, whose developer tier is free up to 5 Data Assets under test per month and 3 users — but GX Cloud publishes tier structure, not dollar prices, so confirm current terms before budgeting.

Why do old tutorials start with great_expectations init?

Because they were written for GX 0.x. That CLI workflow died with GX 1.0 in August 2024, which replaced it with the fluent Python API (gx.get_context(), context.suites.add, Batch Definitions instead of batch_request). The official retirement blog also notes 0.18 and earlier were sunset in GX Cloud on October 1, 2025, with the older Custom Expectations implementation and Value-Based Batches removed. Any tutorial teaching the CLI path is teaching a dead workflow.

What is the difference between Great Expectations and Soda?

Soda is SQL/YAML-based, easy to set up, and built for real-time monitoring with templated checks and alerting. Great Expectations is Python-based, more flexible, and built for complex validations and documentation through Data Docs. Both have free open-source versions — Soda Core and GX OSS — and Soda also offers the commercial Soda Cloud and Soda Agent products.

Does Great Expectations replace dbt tests?

No — they guard different layers. dbt tests live inside your transforms and check the models you build. GX Expectation Suites guard the data contract: what the data must look like regardless of which tool produced it, validated at the pipeline gate before bad data moves downstream. Most teams that use both keep them in their separate lanes.

Does Great Expectations work on Windows?

Not currently for GX OSS — the official quickstart lists Windows support as unavailable, so Windows users should run it under WSL or on Linux. The docs also disagree with each other on the exact supported Python version range, so check the current supported-versions page rather than trusting a version number from any single source.