How to Use DataHub: The Beginner's Guide

CodexFuse Team · 2026-11-03T12:29:28.580Z · 15 min read min read

A DataHub for beginners guide: install the CLI, run the Docker quickstart, log in, load the sample data, write your first ingestion recipe, read the lineage map, and learn the UI-versus-CLI rule — with commands verified against the official docs.

If your data team keeps asking the same three questions — where is that table, who owns it, and what breaks if I change it — this DataHub tutorial is for you. Learning how to use DataHub as a beginner means learning one tool that answers all three: the open-source metadata platform that catalogs every dataset, dashboard, and pipeline in one searchable place, complete with ownership, documentation, and lineage maps.

This DataHub for beginners guide walks the complete first arc: install the CLI, run the Docker quickstart, log in, load the sample data, write your first ingestion recipe against a real source, read the lineage map, and learn the UI-versus-CLI rule that decides which ingestion path to use. Every command is from the current official docs, verified in November 2026. One disambiguation before we start: this is DataHub the open-source metadata platform (docs.datahub.com) — not Alibaba Cloud's DataHub service, not UC Berkeley's JupyterHub "DataHub", and not MarkLogic's Data Hub. The generic search page mixes all four, and most beginner confusion starts there. If you are the one building the tables DataHub will catalog, read the dbt beginner guide first — DataHub's dbt ingestion source pulls your models, tests, and lineage straight from the manifest dbt builds.

What Is DataHub, Exactly?

DataHub is an open-source data catalog and metadata platform. It gives every dataset, dashboard, pipeline, user, and glossary term a searchable home with ownership, documentation, and lineage — the "who owns it, what does it mean, and where did it come from" layer that sits above your warehouse.

The core mental model is the entity-aspect graph, and the official concepts page states it plainly: "An entity is the primary node in the metadata graph. For example, an instance of a Dataset or a CorpUser is an Entity." And: "An aspect is a collection of attributes that describes a particular facet of an entity. Aspects can be shared across entities, for example 'Ownership' is an aspect that is re-used across all the Entities that have owners." DataHub is Kafka/event-driven and streaming-first in its architecture, and the open-source core (DataHub Core) is free under Apache 2.0 — its GitHub repository carries roughly 12,722 stars as cataloged by Atlan, with LinkedIn, Netflix, and Uber running it at scale. The company behind the project (founded 2021, Palo Alto) sells a fully managed DataHub Cloud, but this guide uses only the free self-hosted core.

What Do I Need Before I Start?

You need three things: Docker Desktop with the engine running, Python 3.10 or newer inside a virtual environment if you want the CLI, and a free local port 9002. The quickstart deploys the whole stack — GMS, Kafka, OpenSearch, MySQL — via docker-compose, so give Docker a few gigabytes of disk and a few minutes on first pull.

One warning that belongs before any command, quoted verbatim from the official quickstart: "Quickstart is not intended for a production environment. We recommend deploying DataHub to production using Kubernetes." And: "The quickstart method of running DataHub is intended for local development and a quick way to experience the features that DataHub has to offer." Everything in this guide is a learning install — never a deployment pattern.

How Do I Install DataHub?

Install the DataHub CLI first, then let it deploy the stack for you. The official CLI docs require Python 3.10+ in a virtual environment, and the install is:

python3 -m pip install --upgrade pip wheel setuptools
python3 -m pip install --upgrade acryl-datahub
datahub version

Note the package name: acryl-datahub, not datahub — the datahub name on PyPI is something else entirely, and this is the single most common install mistake. macOS and Linux users can also install via Homebrew with brew install datahub-project/tap/datahub. If you are rusty on containers, the Docker beginner guide covers the engine the quickstart runs on.

With the CLI installed, launch the quickstart. The official docs' instruction, verbatim: "Run the following CLI command from your terminal. datahub docker quickstart — This will deploy a DataHub instance using docker-compose." The first run pulls the images and starts every service; later runs just start what is already there.

What Does the First-Run Arc Look Like?

Your first session has six steps, and the order is the lesson:

1. Install the CLI — pip install acryl-datahub inside a Python 3.10+ virtual environment, then datahub version to confirm.
2. Launch the stack — datahub docker quickstart deploys DataHub via docker-compose.
3. Log in — open http://localhost:9002 and sign in with username datahub / password datahub.
4. Load sample data — run datahub init, then datahub datapack load showcase-ecommerce to fill the catalog with a realistic demo.
5. Explore — search for a dataset, open its profile, and click its Lineage tab to see the graph.
6. Ingest something real — write your first YAML ingestion recipe and run datahub ingest -c recipe.yml.

Notice what comes first and what comes last: the CLI and the stack come first, your own data comes last. The sample data exists so you learn the UI on a catalog that already has answers in it.

How Do I Log In and Explore the Sample Data?

When the quickstart finishes, the docs say verbatim: "Upon completion of this step, you should be able to navigate to the DataHub UI at http://localhost:9002 in your browser. You can sign in using the default credentials below. username: datahub / password: datahub." Change these before anything shared touches the network — they are demo credentials, not a security posture.

The quickstart also tells you exactly how to populate the catalog: "Load sample data: run datahub init then datahub datapack load showcase-ecommerce, or head to http://localhost:9002 (username: datahub, password: datahub) to play around with the frontend." The showcase-ecommerce datapack loads a rich set of roughly 1,050 entities across Snowflake, Looker, PowerBI, and Tableau — with lineage, governance, glossary terms, domains, and data products. One caveat, from the docs: "The datahub datapack command is currently experimental." Older guides teach a legacy sample-data command that the official docs have since replaced with this datapack — treat those old commands as superseded.

Start exploring by searching for anything ("orders", "revenue", a table name) and opening a dataset profile: schema, ownership, tags, glossary terms, and the Lineage tab. If your dashboards live in a BI tool, the Metabase beginner guide is the companion read — your catalog's dashboard entities are someone else's BI workspace, and thinking of them that way is the first governance instinct.

How Do I Ingest My First Real Table?

DataHub ingestion runs on recipes, and the official recipe overview gives the rule in one paragraph: "A recipe is a YAML file that configures one metadata ingestion job. It names a source (the system metadata is read from) and a sink (where that metadata is written, usually DataHub). Run it with datahub ingest -c recipe.dhub.yaml." The second rule follows immediately: "A recipe contains exactly one source and one sink. To ingest from several systems, write several recipes, and give each one its own pipeline_name."

The official docs' minimal example names Snowflake as the source. Adapted here with placeholder credentials — never paste real database credentials into a recipe file you might commit:

# my_first_recipe.yml  (placeholders — use your own secrets store)
source:
  type: snowflake
  config:
    account_id: my_account
    username: my_user
    password: my_password
    role: DATAHUB_ROLE
    warehouse: COMPUTE_WH
sink:
  type: datahub-rest
  config:
    server: http://localhost:8080
pipeline_name: snowflake_prod_first_run

Run it with datahub ingest -c my_first_recipe.yml. Connector plugins install separately from the core CLI — for MySQL, for example: pip install --upgrade 'acryl-datahub[mysql]'. The dbt source is the one data teams reach for first: it reads your dbt manifest and brings in models, tests, and lineage without you re-describing anything.

The UI ingestion page documents one-click templates for Snowflake, BigQuery, Redshift, Databricks, MySQL, PostgreSQL, Looker, Tableau, PowerBI, and Kafka — which is why most beginners never hand-write more than one or two recipes.

Should I Use the UI or the CLI for Ingestion?

This is the core workflow lesson, and the official docs draw the line for you. The UI path: "navigate to the Ingestion tab in DataHub… Begin by clicking + Create source… select the type of data source…" — then fill in the connection form, use the Test Connection button, store credentials under Ingestion → Secrets, set a cron sync schedule, and hit Save or Save and Run. UI sources require the Manage Metadata Ingestion privilege (the Admin role has it by default).

The docs' own comparison table calls UI ingestion "One-click setup, no code required" and the CLI + YAML route "Scheduled batch ingestion, CI/CD pipelines." The advanced cases that push you to YAML are quoted verbatim: "Custom ingestion sources not available in the UI; Complex transformation pipelines; Advanced filtering and processing logic; Integration with external systems." The two paths are not rivals: the CLI can even deploy a scheduled source into the UI with datahub ingest deploy --name "My Test Ingestion Source" --schedule "5 * * * *" --time-zone "UTC" -c recipe.yaml.

One rule above all: ingestion is a habit, not an event. A catalog whose sources never re-run goes stale within weeks, and a stale catalog is worse than no catalog — it teaches your team to distrust it. Set the schedule at creation time, whichever path you use.

How Do I Read the Lineage Map?

The official DataHub lineage docs open with the definition to use: "Data lineage is a map that shows how data flows through your organization. It details where your data originates, how it travels, and where it ultimately ends up." You view it "under the Lineage tab, in an Explorer visualization or the Impact Analysis tool."

Lineage in DataHub is richer than most beginners expect: it captures both table-level and column-level flow — "Column-level lineage tracks changes and movements for each specific data column… Column lineage can be visualized while viewing table-level lineage by expanding the columns of tables and hovering or clicking on columns with lineage." It crosses platforms (Snowflake tables through dbt into Looker dashboards), covers data-pipeline task lineage, and supports manual editing for Datasets, Charts, Dashboards, and Data Jobs. Sources with the "Table Lineage Capability" extract it automatically; the docs publish the full auto-extraction matrix.

The practical read: open a dataset, click Lineage, and follow one column upstream to its source and downstream to every dashboard that reads it. That two-direction walk is the answer to "what breaks if I change this" — and it is the single most valuable five minutes in a new DataHub install. Lineage shows you what flows where; the Great Expectations beginner guide covers the companion layer that stops bad data mid-flow before the lineage carries it further.

DataHub vs OpenMetadata: Which Should I Pick?

DataHub and OpenMetadata are the two open-source catalogs that come up in every comparison, and they approach the problem from different architectures. The catalog comparison table below is built from Atlan's open-source catalog survey and the Ovalege survey of AI-powered catalogs:

For most teams the decision is not philosophical: if you already run Kafka-scale infrastructure and want a streaming metadata platform, DataHub's model fits; if you want one deployable with built-in profiling and the widest connector list, OpenMetadata's all-in-one shape is simpler to stand up. Both are Apache 2.0 open source, so trying each costs an afternoon, not a contract.

Five Beginner Mistakes to Avoid

Almost every DataHub beginner trips on one of these, and most trace back to skipping the docs' own warnings:

What Is New in DataHub Right Now?

DataHub is moving fast enough that version numbers matter. The current headline release is v1.8.0, with a release availability date of October 22, 2026 (per the datahub-project GitHub releases page): OpenSearch 3.x support, Metrics and Semantic Models, a dark mode beta, and structured properties. A v1.6.0.4 patch also shipped in October 2026 for the v1.6.0 LTS line with CVE fixes — if you follow a third-party guide that still says "v1.7.0.1", it is stale.

On the commercial side, DataHub Cloud Context reached general availability on October 28, 2026 (Palo Alto, via Business Wire): it turns context management "from a one-time setup into a system that continuously proves and improves agent accuracy", shipping Domain-scoped Context Management, Cloud Context Evals, and Agent Feedback via MCP after a five-month beta with financial-services, retail, and tech firms. None of this changes the self-hosted core this guide installs — but it explains where the project's momentum is going.

Sources

The official DataHub quickstart (docs.datahub.com/docs/quickstart/, fetched 2026-11-03) documents the datahub docker quickstart command, the localhost:9002 login with datahub/datahub credentials, the showcase-ecommerce datapack (~1,050 entities, experimental), and the quickstart-is-not-production warning; the official CLI docs (docs.datahub.com/docs/cli/, fetched 2026-11-03) document the acryl-datahub pip install on Python 3.10+ and the Homebrew tap; the official metadata-ingestion docs (docs.datahub.com/docs/metadata-ingestion/, fetched 2026-11-03) document the Snowflake recipe example and datahub ingest -c; the official recipe overview (docs.datahub.com/docs/metadata-ingestion/recipe_overview/, fetched 2026-11-03) documents the one-source-one-sink rule and per-system pipeline_name convention; the official UI ingestion docs (docs.datahub.com/docs/ui-ingestion, fetched 2026-11-03) document the + Create source flow, Test Connection, the Ingestion → Secrets store, cron scheduling, the Manage Metadata Ingestion privilege, and the UI-vs-CLI comparison; the official lineage docs (docs.datahub.com/docs/features/feature-guides/lineage, fetched 2026-11-03) document the lineage definition, the Lineage tab / Explorer / Impact Analysis views, and table- plus column-level capture; the official concepts page (docs.datahub.com/docs/what-is-datahub/datahub-concepts, fetched 2026-11-03) documents the entity and aspect definitions; the datahub-project GitHub releases page (fetched 2026-11-03) documents v1.8.0 (October 22, 2026) and the v1.6.0.4 LTS patch; Atlan's LinkedIn DataHub tutorial (atlan.com/linkedin-datahub-tutorial/, fetched 2026-11-03) documents the competitor's Docker install walkthrough; Atlan's open-source catalog survey (atlan.com/open-source-data-catalog-tools/, fetched 2026-11-03) documents the 12,722 vs 15,235 GitHub star counts and the Kafka/event-driven vs all-in-one architectural contrast; the Ovalege AI-powered catalogs survey (ovaledge.com/blog/ai-powered-open-source-data-catalogs, fetched 2026-11-03) documents OpenMetadata's 130+ connectors and built-in data-quality profiling; the sider.ai end-to-end DataHub guide (sider.ai/blog/ai-tools/how-to-use-datahub-a-practical-end-to-end-guide-for-your-data-catalog, fetched 2026-11-03) corroborates the quickstart → ingest → lineage beginner arc; the DataHub Cloud Context GA announcement (datahub.com/news/datahub-cloud-context-general-availability/ and Business Wire, October 28, 2026, fetched 2026-11-03) documents the GA date, the three GA capabilities, the company founding (2021, Palo Alto), and named customers including Apple, Block, Chime, Etsy, Foursquare, FIS, Netflix, and Notion.

Frequently Asked Questions

Is DataHub free to use?

Yes — DataHub Core is open source under Apache 2.0 and free to self-host. The company also sells DataHub Cloud as a fully managed offering ("Try DataHub Cloud free," per the docs), but nothing in this guide needs it.

Is this the same DataHub as Alibaba Cloud's DataHub?

No. This guide covers DataHub the open-source metadata platform at docs.datahub.com (originally built at LinkedIn). Alibaba Cloud's DataHub is a separate streaming-data service, UC Berkeley's "DataHub" is a JupyterHub deployment, and MarkLogic's Data Hub is a different product — generic search results mix all four.

Can I do everything in the UI, or do I need the CLI?

The UI handles most common ingestion scenarios — one-click source setup, connection testing, secrets, and cron scheduling. You need the CLI and YAML recipes for custom sources not in the UI templates, complex transformation pipelines, advanced filtering, and CI/CD-driven ingestion. Many teams use both: recipes in version control, deployed to the UI schedule.

How is DataHub different from OpenMetadata?

DataHub is a Kafka/event-driven, streaming-first metadata platform with a flexible entity-aspect schema and deeper large-scale production use (LinkedIn, Netflix, Uber), at the cost of more infrastructure complexity. OpenMetadata is a leaner all-in-one platform with 130+ connectors and built-in data-quality profiling, with a younger governance model. Both are Apache 2.0.

Does DataHub replace dbt?

No — they do different jobs. dbt transforms your data and tests the models it builds; DataHub catalogs everything (including dbt's own models, via the dbt ingestion source), tracks ownership and lineage, and governs the metadata layer. Most teams that use both keep them in separate lanes.