What the Google Data Catalog shutdown means for teams still on it
Data Catalog was discontinued on June 1, 2026, so any team still reading from it, writing to it or relying on its API has to move. Google's path is Knowledge Catalog, the name Dataplex Universal Catalog took in April 2026. If the catalog only needs to stay inside Google Cloud and Google's managed service meets your residency and access rules, finish the Knowledge Catalog move. If the catalog must run in your own infrastructure, or must hold metadata from beyond Google Cloud under your own control, move to a self-hosted open-source catalog such as OpenMetadata or DataHub. Both paths are valid, and the right one depends on where your metadata must live and who must be able to operate it. The constraint is a date, not a preference. Any integration, scheduled job or user workflow that still points at Data Catalog stops working after that shutdown, so the first step is an inventory: list every consumer of the old API, every saved search, every tag template and every policy tag reference that depends on it. Confirm with Google which of these are carried into Knowledge Catalog and which need rebuilding, because the answer varies by object type. Treat the inventory as the input to both options, since it sets the migration effort whichever catalog you choose. Whichever way you go, decide early what happens to BigQuery policy tags. Policy tags drive column-level access control, and a catalog migration should not change who can read sensitive columns. Confirm with your Google Cloud team that the policy taxonomy stays in Google Cloud and remains the enforcement point, and plan any open-source catalog as a discovery and governance layer that reflects those tags rather than replacing them. Document that split in your data governance design before cutover.
What leaving Data Catalog requires from your metadata
Moving off Data Catalog is less about copying objects than about carrying over the metadata people curated by hand. Each requirement below is a decision or check you can complete before the new catalog goes live. Start with the inventory, because every later step depends on knowing what exists.
Inventory entries and curated fields
List every entry in scope by project, entry group and dataset, then record which curated fields sit on each one: descriptions, owners, glossary links and tags. Mark the hand-written fields separately, since an automated crawl will not recreate them. Decide which fields move, which are retired and who approves each decision. Count entries per project so you can check the export against that baseline later.
Export that keeps the fields
Confirm that the metadata export returns the curated fields you inventoried, not only resource names and schemas. Test it on one project before a full run, then compare a sample of entries field by field against the console. Check where the output lands in Cloud Storage, who can read it and how long it stays there, since it contains schema and ownership data. Treat the verified export as the source of record for the migration.
Target that ingests your sources
List every system that holds data you catalog, including BigQuery, Looker, databases outside Google Cloud and transformation tools. Then confirm each source has a connector or integration in the target, and note any gap that needs a custom script or manual entry. OpenMetadata lists more than 130 connectors and DataHub lists more than 150 integrations, but list counts do not prove your source version is covered. Test one connector per source type before committing.
Where the catalog runs
Decide whether the new catalog runs in the same Google Cloud project, in another account or in your own data center. OpenMetadata documents production deployments on Google GKE, Amazon EKS, Azure AKS and on-premises Kubernetes, and DataHub runs on Kubernetes with Kafka, a relational database and Elasticsearch. Each choice sets the network path to BigQuery and other sources, the egress you pay for and who operates the cluster. Confirm sizing and an on-call owner before scheduling cutover.
Identity and access to the catalog
Map who can read, edit and administer catalog entries today, then rebuild those roles in the new catalog. DataHub supports OpenID Connect single sign-on, but on self-deployed DataHub Core your team configures the frontend server itself. Confirm group mapping from your identity provider, and decide who may change glossary terms and ownership. Check OpenMetadata's security options separately before you assume parity.
BigQuery policy tags and enforcement
Identify which Data Catalog tags correspond to BigQuery policy tags that restrict column access, and keep those enforcement settings in BigQuery. A tag in the new catalog is metadata, so access to BigQuery data stays controlled by what BigQuery enforces. Confirm how each target product's classifications connect to any access control before relying on them. Document the mapping from each tag to its source so auditors can trace it.
When finishing the move on Knowledge Catalog is the right call
Finishing the move to Knowledge Catalog is the simpler choice when most of your data estate already lives in Google Cloud. Data Catalog was discontinued on June 1, 2026, and Knowledge Catalog, the renamed Dataplex Universal Catalog, is Google's successor. For a team whose BigQuery datasets, pipelines and identity model sit inside Google Cloud, the work is mostly migration of entries, tags and access settings rather than a new platform. You keep the same IAM roles, audit logging and project-level administration your cloud team already runs, and you avoid standing up Kubernetes, Kafka and a search cluster that a self-hosted catalog would need.
The trade-off is that the catalog stays a managed Google service. Your metadata model, search behavior and export options are set by Google's roadmap, and a future change to pricing, features or retention is outside your control. If you choose this path, have the team document which Data Catalog objects you still depend on, such as tag templates, search links from other tools and API calls in scripts, and map each one to its Knowledge Catalog equivalent before the cutover date passes.
Before you commit, confirm current Knowledge Catalog feature coverage, including glossary support, tag and policy tag handling for BigQuery, and the export behavior described in Google's documentation for the metadata export jobs. Ask Google or your account team for written confirmation of any feature your governance process requires, and test a full export and re-import in a non-production project. If that test shows gaps you cannot accept, the open-source options on this page become the fallback rather than a surprise later.
Knowledge Catalog, OpenMetadata or DataHub: how each meets the exit
This table compares Knowledge Catalog, OpenMetadata and DataHub on the factors that matter when leaving Data Catalog: who operates the service, connector reach beyond Google Cloud, lineage depth, license and day-two effort. Read the operating burden row first if you need a catalog you control, and the connector row first if your data sits across several clouds or on-premises.
| Criterion | Knowledge Catalog (Google) | OpenMetadata (self-hosted) | DataHub Core (self-hosted) |
|---|---|---|---|
| Who runs the service | Google Cloud manages the catalog service as part of Google Cloud | Your team runs it on Kubernetes, bare metal or Docker for evaluation | Your team runs it on Kubernetes with Helm charts or Docker quickstart for evaluation |
| Connector coverage | Automatic cataloging of Google Cloud resources, with custom entries or connectors for other sources | More than 130 connectors for warehouses, BI tools, pipelines and ML platforms | More than 150 integrations, including Snowflake, BigQuery, Databricks, dbt, Looker and Tableau |
| Lineage | Confirm current lineage scope for your Google Cloud resources with Google | Table, column, dashboard, pipeline and metric lineage, with OpenLineage event integration | Column-level lineage listed among core features in the project README |
| License | Commercial Google Cloud service, billed under your Google Cloud agreement | Apache License 2.0 | Apache License 2.0 |
| Upgrades and patching | Handled by Google as a managed service | Your team applies releases; 2.0 includes breaking changes to review before upgrading | Your team applies releases; the Helm chart runs a system-update job for schema changes and search reindexing |
| Core dependencies you operate | None on your side | Database (MySQL or PostgreSQL), Elasticsearch or OpenSearch, and Java 21 for bare metal | Kafka, MySQL, PostgreSQL or MariaDB, Elasticsearch or OpenSearch, and a graph index on Neo4j or Elasticsearch |
| Single sign-on | Managed through Google Cloud identity | Confirm SSO options for your identity provider in the security documentation | OpenID Connect; on DataHub Core your operator configures the frontend server |
| Main gap for this exit | Less control over where metadata is stored and how the service is operated | Search and database capacity planning and upgrade testing fall to your team | Four-plus services to run (Kafka and search included), so the operating burden is higher than a single-service deployment |
Choosing OpenMetadata or DataHub for your environment
OpenMetadata documents production deployment on Kubernetes with Helm charts, on Amazon EKS, Azure AKS, Google GKE or on-premises Kubernetes clusters. A team leaving Dataplex on Google Cloud may find GKE the closest match to its current estate, but the cluster choice should follow where the catalog must reach sources and where metadata may be stored. Docker Compose is documented for evaluation, so a production plan starts with the cluster design.
- Cluster target: Choose EKS, AKS, GKE or an on-premises cluster based on network reach to your sources, and confirm the chart and Kubernetes version pairing in the deployment documentation for the release you plan to run.
- Stateful dependencies: Treat the database and search service as persistent components with their own storage, backup and restore tests, because the catalog's metadata lives there and not in the web pods.
- Ingestion workloads: Plan connector runs so that source pulls, such as BigQuery ingestion, do not compete with the UI and API for cluster resources.
- Network and identity: Decide how users reach the catalog, how it reaches private sources and how users sign in before go-live, since these shape ingress, DNS and certificate work.
- Upgrade rehearsal: Test 2.0 chart and image changes in a non-production namespace first, since the 2.0 release notes list breaking changes.
Confirm the supported Kubernetes versions, chart version and resource requests for your chosen release with the OpenMetadata deployment documentation before you size the cluster.
Bare metal removes the Kubernetes control plane from the design, but it moves patching, scaling and failover onto your own operations team. The bare metal documentation lists Java 21, MySQL 8.0.42 or later or PostgreSQL 15 or later, and Elasticsearch 9.x or OpenSearch 3.x as prerequisites. The documented minimum server sizing is 2 vCPUs, 4 GB RAM and 20 GB storage, with the database and search services each needing at least 2 vCPUs, 2 GB RAM and 30 GB storage. Treat these as floors rather than production sizing.
- Runtime: Install Java 21 on the application hosts through your standard image process, and track its patch cycle as you would any other runtime.
- Database: Use MySQL 8.0.42 or later, or PostgreSQL 15 or later, and prove backups with a restore test, not only a backup job log.
- Search: Size Elasticsearch 9.x or OpenSearch 3.x separately from the database, since search memory and storage grow with the amount of indexed metadata.
- Sizing: Add capacity above the documented minimums for the application, database and search hosts, using your own asset count and profiling schedule as the basis.
- 2.0 breaking changes: Review the move of embedding credentials into an llmConfiguration block, the new Databricks auth format, the rebuild of custom ingestion images for Python 3.12, and the migration of Great Expectations to 1.3 before upgrading.
Confirm the exact supported database and search versions for your OpenMetadata release on the bare metal page before you order hardware or set a build standard.
DataHub's Kubernetes guide lists four main components: GMS, the MAE Consumer, the MCE Consumer and the Frontend. GMS is the metadata service that the frontend, ingestion jobs and API clients call. The two consumers read metadata change streams from Kafka, and the Frontend is the user interface. The dependencies are Kafka for streaming metadata changes, a MySQL, PostgreSQL or MariaDB database, Elasticsearch for search and a graph index on Neo4j or Elasticsearch.
- GMS: Plan capacity and network paths around GMS first, since it sits between the user interface, ingestion and API traffic.
- MCE and MAE consumers: Run them as separate workloads and monitor their lag, because a lagging consumer delays how quickly changes appear in search and in the interface.
- Frontend: Treat the Frontend as the sign-in entry point and carry its OIDC configuration into your access design, covered in the next tab.
- Kafka, database and search: Assign an owner to each dependency for backup, patching and capacity, since DataHub cannot run without any of them.
- Graph index: Choose Neo4j or Elasticsearch for the graph index only after confirming which one your chart version and your team can operate.
Confirm the component list and supported dependency versions against the current Kubernetes guide, because the stores and graph index are choices you make at install time.
DataHub is deployed with Helm charts from the acryldata/datahub-helm repository, together with a separate prerequisites chart for Kafka, the database, search and graph dependencies. The Kubernetes guide says more than 7 GB of RAM is needed to run DataHub and its dependencies on Minikube, which is a trial figure and not a production estimate. Single sign-on works through OpenID Connect providers such as Google, Okta and Azure AD. On self-deployed DataHub Core, the operator configures the frontend server, while DataHub Cloud enables OIDC SSO through its interface.
- Two charts: Install the prerequisites chart and the DataHub chart as separate releases, and pin both versions in your change records.
- Trial memory: Use the more-than-7 GB figure only for a Minikube trial, and size production clusters from your own ingestion and user volumes.
- Upgrade job: Schedule a maintenance window for the system-update job, which applies schema changes and reindexes search when needed, so upgrades may take longer than a simple pod rollout.
- OIDC on Core: Configure the frontend server with your identity provider's client settings and the redirect URLs for each environment yourself, since no interface setting does this for you on Core.
- Claims and roles: Confirm which group or role claims your provider sends, and how they map to DataHub access, before pilot users sign in.
Check the OIDC configuration guide for the frontend settings your DataHub version expects before you build the sign-in path, and test it with a non-production identity provider application.
Both projects publish maintenance releases often, so the version you plan against can change before go-live. OpenMetadata 2.0.0 was released on August 24, 2026, with maintenance releases 2.0.1 to 2.0.5 following about every one to two weeks. The project's GitHub releases page records 2.0.5 on October 8, 2026. DataHub 1.7.0.1 was released on September 3, 2026, and v1.8.0 release candidates followed in October 2026, according to DataHub's GitHub releases page.
- OpenMetadata 2.0.5: Use the release recorded on October 8, 2026 as the reference point for your 2.0 upgrade review, and confirm whether a later patch exists.
- DataHub 1.7.0.1: Use the release recorded on September 3, 2026 as a reference point, and check whether the v1.8.0 candidates have become a stable release before you pin.
- Release cadence: Freeze a version at design time, then schedule a re-check of the release pages before procurement and again before cutover.
- Charts and prerequisites: Confirm that the Helm chart, container images and Java or Python prerequisites match the release you choose.
- Support status: Ask Collate, for OpenMetadata, or the company behind DataHub, for its support terms; open-source release dates alone do not define your support window.
Check the GitHub releases pages for both projects on the day you start planning, and record the exact version you tested.
How to run the exit without losing curated metadata
This sequence moves the curated layer of your Dataplex or Knowledge Catalog estate, including descriptions, glossary terms, ownership and lineage, into the catalog you choose, and proves each piece arrived before users switch over. The exact steps depend on whether you complete the Knowledge Catalog path or move to OpenMetadata or DataHub, and on your own environment.
Inventory entries and curated fields
Before exporting anything, list what humans added by hand: descriptions, glossary links, owners, tags, aspects, and any BigQuery policy tags that drive access. Separate these from fields that the catalog harvests automatically, since automatic metadata can be re-ingested from the source. Assign an owner to the inventory and agree on which fields are authoritative. The step is done when you have a written field list per entry group and a named owner who signs off on it.
Run the metadata export job to Cloud Storage
Run a Knowledge Catalog metadata export job so the catalog contents land in Cloud Storage as JSONL files. Confirm that the output bucket is in a location your data residency rules allow and that the service identity has write access. Keep the export as an immutable snapshot with the date in its path, so later edits do not change the baseline you validate against. The step is done when the export completes and you have checked file counts against the inventory from the previous step.
Decide scope: organization, project or entry group
Choose the export scope that matches how your estate is organized. An organization-level scope covers everything but may include entries you no longer want to migrate, while project or entry group scopes let you move in batches and test each batch separately. Document the scope decision with the reasons, since it determines which BigQuery datasets and other sources need connectors in the target. The step is done when the scope is written down and approved by the data owners who depend on those entries.
Load a test environment with the chosen catalog's connectors
Stand up a non-production OpenMetadata or DataHub instance in your own infrastructure and configure connectors for BigQuery and any other sources in scope. Use the exported JSONL as the source for curated fields, transforming it into the target's model, and note every field that has no direct equivalent. For OpenMetadata, check how glossaries and custom properties map. For DataHub, check how entry groups and glossary terms map. The step is done when the test instance holds the full scoped inventory and you have a list of unmapped fields with decisions for each.
Validate lineage and glossary terms against a sample
Pick a representative sample that includes high-use tables, entries with lineage, and entries with glossary terms or ownership. Compare each sampled entry in the target with the export: descriptions, owners, tags, glossary links and lineage edges should match or have a documented reason for differing. Ask the data owners to review their own entries. Confirm that BigQuery policy tags are still enforced at the source, because catalog metadata alone does not control access in BigQuery. The step is done when the sample passes review and every discrepancy is either fixed or accepted in writing.
Cut over users and single sign-on
Configure single sign-on for the target catalog using your identity provider, following the OIDC setup for your chosen platform, and test login with a small group of users first. Move the catalog's URL and any integrations, then announce the cutover with a read-only period for Knowledge Catalog if you are keeping it in parallel. Keep the export snapshot and a rollback plan until owners confirm their entries are correct. The step is done when a pilot group has signed in, searched and edited entries in the new catalog without access errors.
What evidence to keep for the catalog change
An auditor or internal risk team will ask how you know the new catalog matches the old one, who could change it, and where it runs. The cards below cover six records that answer those questions. Keep them together in one change file so the path from the Data Catalog inventory to the target deployment can be followed end to end.
Catalog inventory before and after
Take a count and a list of entries from Data Catalog and from Knowledge Catalog before cutover, then the same from the target catalog after ingestion. Compare entry names, types and owners by project and dataset, not only totals. Record which entries were intentionally excluded, such as retired tables or test datasets, and who approved each exclusion. This is the record that shows nothing was dropped silently.
Export job records and scope
Keep the metadata export job configuration, its scope (organization, project or entry group), the run timestamps and the Cloud Storage output locations. Note the JSONL output files and any transformation scripts used to map them into OpenMetadata or DataHub. Check that the export covers the custom fields and glossary terms your teams rely on, because a default export may not include everything you curated.
Target version and configuration
Record the exact release deployed, such as the OpenMetadata 2.0.5 or DataHub 1.7.0.1 release tags, along with Helm chart versions, the connector list and the configuration files used. Store these in version control so a later reviewer can see what was running on a given date. Keep the upgrade notes you followed, including any breaking-change steps applied before upgrading.
Access control and single sign-on settings
Document who holds administrative rights in the catalog, how roles map to data domains, and the OpenID Connect provider settings used for sign-in. For a self-deployed DataHub Core, the frontend configuration is your responsibility, so save it with the change file. Confirm which group memberships grant read access to metadata that reflects sensitive datasets, and record the review of those mappings.
Backup and restore test results
Keep the backup schedule for the catalog database and search index, the location and retention of each backup, and the dated result of at least one restore into a separate environment. Metadata stores in OpenMetadata and DataHub depend on their database and search services, so record both. Note what a restore does not recover, such as data that was only in the source system and not yet ingested.
Hosting location decision record
Write down where the catalog runs, which cloud account, region or data center hosts it, and why that location was chosen against data residency rules and the sources it reads. Record who approved the decision and the date. If the catalog connects to BigQuery across projects, note the network path and credentials used. This record explains the design to reviewers who did not take part in the migration.
Questions buyers ask before moving off Data Catalog
How much effort does moving off Data Catalog take?
The effort depends mostly on three drivers: the number of sources to connect, how many curated fields (descriptions, owners, glossary terms, tags) must be carried over, and how deep the lineage needs to go. A team with a handful of BigQuery datasets and a short list of glossary terms has a very different job from one with dozens of sources, hand-written business definitions and column-level lineage across pipelines. Ask the implementation partner to size the work by these three counts before agreeing on a plan. Each source adds connector configuration, credential handling and validation, so the source count usually matters most in the early phase.
Can our team run the catalog move without outside help?
A team with strong platform engineering can do it alone, but the move touches several systems at once: a Kubernetes or bare metal host, a database, a search service and the ingestion pipelines. Most teams underestimate the metadata export and mapping step, which needs someone who understands both the Google Cloud side and the target model. A fixed-scope engagement lets you buy that specific work, such as connector setup, curated field migration and a runbook, while your staff keep ownership of the platform afterward. Decide which part you want to own before you ask for a quote.
What skills does our staff need to run a self-hosted catalog?
Plan for three skill areas: container or host operations, database administration and search operations. For DataHub on Kubernetes, the guide names four main components (GMS, the MAE Consumer, the MCE Consumer and the Frontend) that need Kafka, a MySQL, PostgreSQL or MariaDB database, Elasticsearch for search and a graph index on Neo4j or Elasticsearch. OpenMetadata on bare metal requires Java 21, MySQL 8.0.42 or later or PostgreSQL 15 or later, and Elasticsearch 9.x or OpenSearch 3.x. Confirm that your team can monitor, back up and patch these services, because the catalog is only as available as its search and database tiers.
What hardware do the catalogs need to start?
For a small evaluation, the Docker quickstart for DataHub requires Docker with 8 GB of RAM and Python 3.10 or later, and the DataHub Kubernetes guide says more than 7 GB of RAM is needed to run DataHub and its dependencies on Minikube. OpenMetadata's minimum server sizing is 2 vCPUs, 4 GB RAM and 20 GB storage, with the database and search services each needing at least 2 vCPUs, 2 GB RAM and 30 GB storage. These are minimums for trials and small estates, not production sizing. Ask your partner to size production from your source count, query volume and retention needs, and to state the assumptions in the quote.
Is Knowledge Catalog the cheaper path compared with self-hosting?
Knowledge Catalog is a managed service, so its cost is billed by Google Cloud and does not include a platform team to run the catalog. Self-hosting shifts cost into infrastructure, the staff time to operate the services and the one-time migration work, and it recurs through upgrades and monitoring. Which path is cheaper depends on your estate: if everything lives in Google Cloud and your team has no appetite for operating search and databases, the managed path may be the better fit. If you have data across clouds or on-premises, or you need the catalog under your own control, self-hosting may justify its operating cost. Compare total cost over the period you plan to keep the catalog, using your own billing data and staffing estimates.
What should a quote for a catalog migration include?
A useful quote lists scope and assumptions in writing before any work starts. Ask for the source inventory and how each source will be connected, the list of curated fields to carry over and where they come from, the lineage depth in scope, the target platform and hosting location, the Kubernetes or bare metal topology, and the handling of BigQuery policy tags. It should also name deliverables such as deployment, configuration, documentation and handover, what is one-time, what recurs (such as retainer support or upgrades), and what the client must provide, including access, staff time and environments. Fixed-scope engagements are typically agreed this way.
How are upgrades handled after the catalog is live?
Upgrades need a plan because the platforms change their schemas and configuration. The DataHub Helm chart runs a system-update job on upgrades to apply schema changes and reindex search indices when needed, so schedule time for that job and for index rebuilds in your change window. OpenMetadata 2.0 introduced breaking changes that you must handle before upgrading, including moving embedding credentials into an llmConfiguration block, a new Databricks auth format, rebuilding custom ingestion images for Python 3.12 and migrating Great Expectations to 1.3. Test each upgrade in a non-production environment, back up the database and search indices first, and confirm the release notes for each version you pass through. DeployOpen can provide upgrades and continuous support as a retainer.
How DeployOpen works on this
DeployOpen plans and deploys catalog exits inside your own cloud accounts, data centers or air-gapped networks. The work starts with a written scope that separates finishing the Knowledge Catalog transition from moving to a self-hosted OpenMetadata or DataHub catalog, so each path is priced and tested against the same agreed deliverables.
How we scope
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before any work starts.
- We inventory which Data Catalog entries, custom aspects, glossary terms and BigQuery policy tags must be kept, moved or retired.
- We confirm with your team which sources, BI tools and Google Cloud projects the new catalog must cover at launch.
How we build
- We deploy OpenMetadata or DataHub in your own environment, using the deployment method your platform team has chosen.
- We configure connectors for BigQuery and other sources, then test ingestion, lineage and access controls against the agreed scope.
- We document the configuration, metadata export steps and any data quality checks so your team can rerun them.
How we hand over
- Handover includes architecture notes, runbooks, configuration files and a record of every migrated or retired metadata field.
- We walk your operators through upgrades, including the system-update steps and breaking-change checks for the chosen release.
- Evidence for compliance readiness is organized for your auditors, who issue any audit reports or certifications themselves.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after handover.
- Retainer scope, response expectations and upgrade cadence are set in writing so your team knows what is covered.
- Ask for a quote built from your written scope, since DeployOpen does not publish a price list.
