Migrate from Microsoft Purview to DataHub

Migrate from Microsoft Purview to DataHub: what moves and who does the work

The migration is mainly rescanning your sources into DataHub and rebuilding the curated metadata, with a bounded export from Purview to carry over what cannot be rediscovered. Most of the work is not moving bytes. Purview has no one-click catalog export, so the glossary terms, owners, classifications and domain structure must be read through the Purview APIs or re-entered from a reviewed inventory. If your renewal, billing change, residency requirement or lock-in concern is what triggered this review, start by listing which of your governed assets depend on curated Purview content and which can simply be ingested again from the source systems. Platform or data engineering teams usually own this work, either as an internal team or with an implementation partner such as DeployOpen working to a written scope. The effort is mostly one-time: standing up DataHub, writing ingestion recipes for each source, mapping Purview collections to DataHub domains and data products, and reviewing ownership and glossary content with the data owners who will maintain it afterward. The recurring cost is the platform itself, which you operate: the GMS, the MAE and MCE consumers, the Frontend, Kafka, the relational database, the search index and the graph index all need upgrades, backups and capacity planning, and DataHub's Helm chart runs a system-update job on upgrades that you should test first in a non-production cluster. Risk concentrates in three places: sources whose metadata lives only in Purview curation (manual descriptions, custom classifications, approval workflows), lineage that Purview captured through a connector you do not replicate in DataHub, and cutover timing while consuming teams still depend on Purview. Before you commit to a timeline, ask whether each source has a DataHub ingestion source, confirm the exact connector configuration in the DataHub Kubernetes guide, and agree with your auditors which catalog records must survive the switch.

Which Purview capabilities map to DataHub, and which need rework

This table maps the main Purview Unified Catalog capabilities to their nearest DataHub Core equivalents and marks each as clean, needs rework or gap. Read the status column first, then the notes on what must be rebuilt by hand. The main takeaway is that discovery, lineage and glossary usually carry over, while data products, domains and DGPU-based data quality rules need deliberate redesign and features such as access request workflows and compliance forms are not in DataHub Core.

Purview capability	DataHub Core equivalent	Status	What to plan
Data Map scanning	150+ ingestion integrations, including Snowflake, BigQuery, Databricks, dbt, Looker and Tableau	Clean	Rebuild each source as a DataHub recipe and confirm connector coverage for every Purview source type in use
Business glossary	Business glossary with terms linked to datasets	Clean	Export terms through the Purview REST APIs and map term hierarchy and definitions to DataHub glossary nodes
Ownership	Ownership on datasets and entities	Clean	Map owner identities to DataHub users and groups, and confirm the identity source used for SSO
Classifications	Tags and structured properties, mapped from Purview classifications	Needs rework	Classification names, rules and scan-time logic do not transfer automatically, so re-create tags and confirm propagation behavior
Lineage	Column-level lineage from supported connectors	Needs rework	Confirm lineage for each source and rerun ingestion, since Purview lineage history is not carried over by a rescan
Data products	Data products in DataHub, with manual assignment	Needs rework	Rebuild each product, its assets and owners by hand from the Purview export
Domains	Domains for grouping assets	Needs rework	Redesign the domain hierarchy and assign assets, because Purview domain structure is not a one-to-one mapping
Data quality rules and DGPU monitoring	Data quality assertions and monitoring in DataHub Core	Gap	Purview DGPU-based rules have no direct equivalent, so rewrite each rule as an assertion and confirm which checks run in your environment
Access requests and compliance forms	Available in DataHub Cloud only	Gap	DataHub Core does not include these workflows, so plan approval processes outside the catalog or confirm the Cloud scope

What to export from Purview and what to rescan instead

Purview metadata reaches DataHub through two paths: curated content exported through the Purview REST and Atlas-compatible APIs, and technical metadata rebuilt by DataHub ingestion. The cards below separate those paths and list the work on each. Settle the split for every object type before the first ingestion run.

Glossary terms and owners

The documented read path out of Purview runs through its REST and Atlas-compatible APIs, which return glossary terms, term assignments and entity owners. Export these as a reviewed file set, because curated business definitions are the content a scan cannot regenerate. Check that each term has its definition, steward and linked assets, and that every owner record maps to someone who still works in your organization. Curated terms are the content to export.

Technical metadata: rescan instead

Schemas, tables, columns and data types are cheaper to rescan than to export. DataHub's ingestion connectors read each source system directly and rebuild those entities, along with usage and profiling where you enable them. Exports and rescans will overlap, so assign one owner per object type before you start. After the first run, compare entity counts with Purview's inventory to find gaps.

Version-control ingestion recipes

Each DataHub source is defined in a YAML recipe that names the source, its connection settings and the sink. Store recipes in Git beside the infrastructure code that deploys DataHub, and reference credentials from the secret store your team already runs rather than writing them into recipe.yml. Review recipe changes like code so a scope change is visible before it reaches production. Confirm the current recipe format for each connector in the DataHub docs before copying any example.

Azure sources and connector gaps

Azure SQL, Synapse, Data Lake Storage and Fabric sources were scanned by Purview, so each one needs a DataHub connector and a service identity with read access. Confirm connector support and version for every Azure source in DataHub's integration list before you schedule work, because the README's count of more than 150 integrations does not name each one. Where a source has no connector, record the gap and decide whether to write a custom source or keep those entities as manually curated metadata.

dbt and BI lineage rebuild

The DataHub README lists dbt, Looker and Tableau among its integrations and names column-level lineage as a feature. Purview lineage for transformations and dashboards often comes from scans of those tools, so rebuild those connections first. Compare the lineage paths for your most-used reports before and after cutover. Confirm that dbt model descriptions and tests carry over, and that dashboard owners match the owners you exported.

OIDC sign-in for DataHub Core

On self-deployed DataHub Core, your team configures OpenID Connect on the frontend server, while DataHub Cloud enables OIDC through its interface. DataHub's guide lists Azure AD among supported providers, alongside Google and Okta. Plan the redirect URI, client secret storage and group-to-role mapping before cutover, and test sign-in with a non-admin account.

Migration sequence from inventory to Purview decommission

This sequence moves governed metadata from Microsoft Purview Unified Catalog into self-hosted DataHub in controlled stages, so each phase leaves a checkable output and Purview is switched off only after the two catalogs agree. Each phase has a named owner and a written exit check.

Build the governed asset inventory
The data governance lead and the Purview administrator export the list of data sources, assets, and collections that Purview currently governs, including which assets carry owners, classifications, or data quality controls. Purview has no one-click catalog export, so this inventory is usually built from Data Map API reads and a review of each source registration. The step is done when the inventory lists every source with its owner, sensitivity labels, and a status of rescan or export, and the platform team has signed off on the scope.
Export glossary terms and ownership
Data stewards pull glossary terms, term-to-asset links, and owner assignments through the Purview REST APIs and store them in a neutral format such as CSV or JSON. They resolve duplicate terms, retire stale ones, and record which entries need a steward decision before import. The output is a curated glossary and ownership file with one row per term or owner link, plus a short list of open decisions. Confirm current endpoint names and paging behavior against Microsoft's Purview documentation for your tenant before scripting the export.
Map domains and data products to DataHub
The governance lead and domain owners translate Purview domains, collections, and data products into DataHub concepts such as domains, data products, and ownership groups, then decide how Purview classifications map to DataHub tags and glossary terms. This mapping workbook becomes the reference for every later import and review. The step is complete when each Purview object has a target DataHub object or an explicit reason to drop it, and each domain owner has approved their rows.
Pilot ingestion on a few sources
Platform engineers choose two or three representative sources, such as a warehouse, a lakehouse, and a BI tool, and write DataHub ingestion recipes for them in version-controlled files. They run the recipes against a non-production DataHub deployment, then compare the ingested schemas, owners, and lineage with what Purview showed for the same sources. The output is a tested ingestion recipe set and a short list of gaps, such as lineage or classification differences, that need a documented workaround or an accepted loss.
Run both catalogs in parallel and reconcile
For a defined period agreed in the project plan, users and stewards work in both catalogs while the DataHub ingestion schedule runs against every in-scope source. A reconciliation job compares asset counts, owners, glossary links, and domain assignments between Purview and DataHub, and the platform team reviews each mismatch. The output is a reconciliation report with every difference classified as fixed, accepted, or deferred, and the sign-off of the data owners for the in-scope domains.
Cut over users and integrations
The platform team points consumers, such as access request workflows, documentation links, and downstream tools that call Purview APIs, to DataHub, and updates the runbook for stewards. Access to DataHub is configured through OpenID Connect single sign-on, which on self-deployed DataHub Core the operator must configure on the frontend server. The step is done when all named consumers have been switched, the rollback path is documented, and the owner of DataHub operations has accepted support responsibility.
Archive the Purview export and decommission
Before the Purview subscription or Data Map resources are removed, the governance lead archives the final API exports, the mapping workbook, the reconciliation report, and the recipe repository in retained storage that meets your records policy. Billing and contract steps are confirmed with Microsoft and procurement, not assumed. The step closes when the archive has been verified by a second person, the retention owner has signed the record, and Purview resources are removed according to the written decommission plan.

Planning DataHub's Kafka, database and search footprint

Before you plan capacity, know what you are deploying. DataHub's Kubernetes guide describes four main components: GMS (the metadata service that the API and UI call), the MAE Consumer and MCE Consumer (which process metadata change events), and the Frontend. These components depend on Kafka for streaming metadata changes, a MySQL, PostgreSQL or MariaDB database for storage, Elasticsearch for search, and a graph index on Neo4j or Elasticsearch. The Helm chart comes from the acryldata/datahub-helm repository, with a separate prerequisites chart for the dependencies. Confirm the current chart and dependency versions in the DataHub Kubernetes documentation before you pin them in your environment.

The documented memory figures are starting points, not production sizing. The Docker quickstart requires Docker with 8 GB of RAM, and the Kubernetes guide says more than 7 GB is needed to run DataHub and its dependencies on Minikube. Both suit evaluation with a sample of sources. For production, size Kafka, the metadata database and the search cluster from your own inputs: the number of Purview-migrated assets and the sources you will rescan, the volume of schema and lineage changes per ingestion run, and how often each source is ingested. Ask your platform team to load-test one full rescan of your largest source before fixing node counts.

Plan for upgrades as well as steady state. The Helm chart runs a system-update job on upgrades that applies schema changes and reindexes search when needed. Reindexing can be heavy on Elasticsearch or OpenSearch, so schedule upgrades in a window where the search cluster has spare capacity, keep a tested backup of the metadata database, and confirm with your operator team how long the job runs on a catalog of your size. Decide early who owns Kafka retention, database backups and search snapshots, since these are the recurring operating tasks that a self-hosted catalog adds compared with a hosted service.

How the migration differs by starting point

When every governed source sits in Azure, the scope is narrow and most of the effort goes into rebuilding ingestion and identity rather than reconnecting networks. Your Purview scans become DataHub ingestion recipes, one per source type, so you can run each recipe and compare entity counts against what Purview reports before you retire the old scan.

Ingestion rebuild: Recreate each Purview scan as a DataHub recipe for the same Azure stores, using the connectors listed in the DataHub repository. Migrate one source at a time.
Identity: DataHub supports OpenID Connect providers, and Azure AD is one of the named examples. On self-deployed DataHub Core, your team configures the frontend server, so schedule that work before user onboarding.
Glossary and ownership: Export curated terms and owners through the Purview REST APIs, since Purview has no one-click catalog export.
Network placement: Place the DataHub services and their database and search tiers so they can reach the Azure sources you scan. Confirm the private connectivity pattern with your network team.
Upgrades: Plan for the Helm chart's system-update job, which applies schema changes and reindexes search when needed.

Before you start, confirm connector coverage for each Azure source type against the current DataHub release and list any source that lacks a maintained connector.

What usually goes wrong in a Purview to DataHub migration

Most failed Purview to DataHub moves do not fail at the first ingestion run. They fail when ownership, domains and access settings are carried over loosely, when sources are left out, or when the self-hosted footprint is sized for a demo rather than production. Each card below names one risk and the check that prevents it.

Ownership and glossary terms lost in translation

Purview owners, stewards and glossary term links do not map one to one onto DataHub owner types and glossary nodes. Export the Purview glossary and entity owner fields through the Data Map APIs, then compare a sample of assets per owner in both systems before cutover. Check that each term still links to the datasets it described, because a term that loads without its relationships looks correct in a demo and fails in audits.

Domains and data products flattened

Purview collections and Unified Catalog domains often carry hierarchy that DataHub represents with domains and data products under different rules. Teams that import everything into one flat domain lose the accountability their data product owners relied on. Write the domain map as a table before ingestion, agree it with each data owner, and confirm that every data product has a named owner in DataHub after the import.

Ingestion gaps in unconnected sources

A rescan covers only the sources you configure, and a source left out of the first wave often disappears from the catalog without anyone noticing. Keep a written inventory of every Purview data source, its scan schedule and its owner, then map each one to a DataHub ingestion recipe. Confirm which connectors cover your sources in the DataHub integrations list before you retire any Purview scan.

Search and graph index sizing failures

DataHub stores metadata in a relational database, indexes it in Elasticsearch or OpenSearch and keeps relationships in a graph index. Undersized search or graph nodes slow discovery and lineage queries long after the import looks successful. Load a representative copy of your largest source metadata into a staging cluster, measure search latency and memory use, and size production from those results rather than from the quickstart requirements.

SSO misconfiguration on DataHub Core

DataHub supports OpenID Connect single sign-on with providers such as Azure AD, but on self-deployed DataHub Core the operator must configure the frontend server by hand. A wrong redirect URI, an unmapped group claim or a missing role mapping can leave users locked out or, worse, grant broad access. Test login with a non-admin account, confirm group-based roles match the Purview permission model, and document the settings for your security review.

Version drift during upgrades

The DataHub Helm chart runs a system-update job that applies schema changes and reindexes search when needed, so an upgrade changes stored data as well as containers. Running ingestion from an older CLI against a newer server, or upgrading without a backup, is a common cause of broken lineage and missing search results. Pin the chart, the ingestion CLI and the server to matching releases, take a database and search snapshot before each upgrade, and track the project's GitHub releases page for changes to your version.

What drives cost and duration, and how to keep each down

What is the biggest driver of cost when moving from Purview to DataHub?

The size and shape of your governed estate drives most of the work, because every governed asset, source and glossary term has to be re-ingested, reviewed and mapped. Purview bills per governed asset and per data quality processing unit, so asset count also shapes what you pay while you run both catalogs. Costs fall into one-time work (rebuild, pilot and cutover) and recurring work (infrastructure, upgrades and support). Ask any partner to show the asset count and source list their estimate is based on, and to separate those two groups of cost.

How does the number of source connectors affect the migration effort?

Each source needs its own DataHub ingestion recipe, credentials, scheduling and validation, so connector count is often the largest driver of rebuild hours. DataHub lists more than 150 integrations, including Snowflake, BigQuery, Databricks, dbt, Looker and Tableau, but coverage for your exact source versions still needs checking. Sources that Purview scans through its own managed mechanisms may need a new network path or service account. Ask for a per-source list stating who builds each recipe and who tests it.

Why do glossary and ownership volume matter more than they seem?

Glossary terms, term-to-asset links and owner assignments are curated by people, so they take review time that scales with volume. Purview has no one-click export, so terms and ownership usually go through the Purview REST APIs and a mapping step into DataHub's glossary and ownership model. Reconciling duplicate terms, stale owners and missing stewards is business work, not only scripting. Ask the provider how many terms and owners are in scope, who signs off each domain, and how many review rounds are included.

What does it cost to rebuild data quality rules in DataHub?

Rebuilding rules is one-time work, because DataHub assertions and monitors are configured separately from Purview's data quality processing. The effort depends on how many rules exist, how many were actually used by data owners, and whether the thresholds still match current business expectations. DataHub Cloud adds data quality monitoring, AI anomaly detection and compliance forms that are not in open source, so the scope of rebuilt rules should be fixed against DataHub Core features. Ask for a rule inventory, a keep, rewrite or retire decision for each rule, and the test method for each one.

How much infrastructure does DataHub need for Kafka, database and search?

DataHub runs GMS, the MAE Consumer, the MCE Consumer and the Frontend, and needs Kafka, a MySQL, PostgreSQL or MariaDB database, Elasticsearch or OpenSearch for search, and a graph index on Neo4j or Elasticsearch. These are recurring costs for compute, storage, backups and patching, and their size grows with metadata volume and ingestion frequency. The Kubernetes guide says more than 7 GB of RAM is needed to run DataHub and its dependencies on Minikube, which is a development floor, not a production sizing. Ask the provider for a sizing based on your asset count, ingestion schedule, retention and high availability targets.

Who pays for staff time, and how much of it is recurring?

Internal staff time is usually the largest cost and it has two parts: a one-time migration effort from data platform, data stewards and security, and a recurring effort to operate the platform. Operating work includes ingestion failures, Helm chart upgrades, the system-update job that applies schema changes and reindexes search when needed, and user access. Your own team, a partner under a fixed-scope engagement, or an ongoing support retainer can cover each part. Ask who owns each task after cutover and what the handover includes.

How long should a parallel run last, and what does it cost to keep both catalogs running?

A parallel run lasts until ingestion, glossary, ownership and rules match in both catalogs and business users confirm search results, so its length depends on the exit criteria you set rather than a fixed period. While both run, you pay Purview consumption and DataHub infrastructure at the same time, so a longer run increases cost. Define cutover criteria, a rollback plan and the date Purview access ends before the run starts. Ask the provider to state the exit criteria and the duration it expects, with the assumptions behind it, in the written scope.

What should I ask for in a written quote for this migration?

A useful quote lists the scope in writing: governed assets, sources, glossary terms, owners, rules, domains and data products in scope, plus the DataHub deployment target and environments. It should separate one-time items (rebuild, pilot, cutover, training) from recurring items (support, upgrades, testing) and state what the client must provide, such as source credentials and network access. Ask for the assumptions behind each line, the acceptance criteria, and what is excluded, including any DataHub Cloud features. DeployOpen works from a written scope and does not publish a price list.

How DeployOpen scopes and runs the migration

DeployOpen is a Chainscore Labs company that plans, implements, integrates, hardens, backs up, upgrades and documents self-hosted open-source platforms inside your own infrastructure, including your cloud accounts, data centers or air-gapped networks. For a Purview to DataHub move, that means the DataHub deployment, its Kafka, database and search dependencies, the ingestion runs against your Azure and other sources, and the export and mapping work for glossary terms, owners and domains all sit in your environment and under your control. Most engagements start with a written scope that names which Purview collections, sources and glossary sets are in play, which DataHub components are deployed, and who on your side approves each step.

Engagements are fixed-scope. The written scope sets out deliverables, acceptance checks and timelines before work begins, so you can see what the migration includes and what it leaves to your team. Typical decisions to settle in that scope include whether DataHub Core is sufficient or whether you need features that exist only in DataHub Cloud, how identity is wired through OpenID Connect single sign-on, and how cutover from Purview is run. Ongoing support, version upgrades and continuous testing are available as retainers, which is useful because DataHub releases and Helm chart changes, including system-update jobs, recur after go-live.

DeployOpen does not publish a price list. A quote follows from the written scope, so the most useful preparation is an inventory of your Purview sources, the number of assets and glossary terms you need to carry over, the infrastructure where DataHub will run, and your recovery and retention requirements. Ask any provider to state which items are one-time (initial deployment, mapping and rescanning) and which recur (hosting, upgrades, support), and to name which parts of the work are their responsibility and which remain with your own staff.

Migrate from Microsoft Purview to DataHub