Why Superset metadata database migrations need their own plan
The metadata database migration is the step most likely to take a Superset platform down or lose data during an upgrade, so it needs its own plan with a verified backup, a staging rehearsal and a rollback point before anyone runs it in production. Application code can usually be redeployed or reverted quickly, but schema changes to the metadata database are applied in place. Once they run, going back to the previous release is not a simple restart.
The supported path in the Superset upgrade guide is to back up the metadata database, run superset db upgrade, run superset init, and test the new release in staging first. The guide also asks you to check UPDATING.md for breaking changes between your current and target versions, because some migrations change or remove data, not just structure. The metadata database is the only stateful component this migration touches. Dashboards, datasets, users, roles and saved queries live there, while the caching layer and Celery workers hold transient state that can be rebuilt.
Superset's architecture documentation names PostgreSQL and MySQL as the tested metadata databases, and the supported versions are listed in the configuration documentation. Confirm your current database engine and version against those lists before you plan, since an unsupported version can turn an upgrade into a database upgrade as well. The SQLite default used by quickstart and PyPI installs is not recommended for production, so a team with a large metadata database should already be on one of the tested engines.
Which migration changes can lock tables or delete rows
This table compares the migration changes that deserve a closer look before running superset db upgrade on a large metadata database. Read each row as a risk type, its impact on data or availability, and the action to take before the upgrade window. The main takeaway is that the 6.1.0 association table migration deletes rows, so an export is needed before that upgrade, and lock behavior should be confirmed in staging for every migration.
| Change | Impact | Action needed |
|---|---|---|
| 6.1.0 composite primary key migration | Deletes duplicate and NULL rows in many-to-many association tables, so data is removed during the upgrade | Export affected rows first if an audit trail is needed, following the UPDATING.md guidance |
| Duplicate and NULL row cleanup in association tables | Rows removed by the migration cannot be recovered from the upgraded database alone | Confirm row counts before and after in staging, and keep the metadata backup until the upgrade is verified |
| Schema migrations on large tables | Alterations to large tables can hold locks that block reads or writes while the migration runs | Confirm lock behavior per migration in UPDATING.md and in staging rather than assume it, and schedule the downtime window accordingly |
| Column type or index changes | Can rewrite or rebuild a table, extending lock time in proportion to table size | Measure duration in a staging copy of production-sized metadata and ask the project which migrations rewrite tables |
| Migrations that run with the web application stopped | Users and scheduled reports fail or queue while the metadata database is in flux | Stop web and worker processes before superset db upgrade and confirm they are stopped before starting the migration |
| Failed or partially applied migration | Leaves the metadata database between schema versions, which can block restart | Restore from the verified backup, then rerun the upgrade in staging to identify the cause before retrying |
How superset db upgrade changes the metadata database
A Superset upgrade changes the metadata database in two steps, and both must finish before users return. The first step, superset db upgrade, applies schema migrations to the metadata database, which is where Superset keeps dashboards, charts, datasource definitions, users and roles. The second step, superset init, then sets up roles and permissions against that updated schema. Treat the two as one operation with a single rollback point, because a database that has been migrated but not initialized is not a state you want users to reach. The upgrade guide asks you to back up the metadata database before either command runs, so confirm that the backup is restorable before you start.
Superset runs as four components: the Flask and React application, the metadata database, an optional caching layer, and optional Celery worker and beat processes. The application and the metadata database are the only components that a migration must touch directly, but the optional parts matter because the architecture documentation says alerts and reports, async queries and dashboard thumbnails depend on the caching layer and workers. A worker that picks up a task while the schema is changing can fail mid-run or write rows the new schema does not expect. Stop or drain the Celery workers and beat before running superset db upgrade, confirm the queues are empty, and restart them only after superset init completes and the web application passes a health check. Confirm with your own deployment team how workers are scheduled in your environment, because a Kubernetes or container setup may restart them automatically.
What to prepare before the migration window
Most failed metadata migrations are not failures of the migration itself. They come from missing backups, untested restores, or nobody owning the decision to roll back. The six items below cover the data, the environment and the people you need in place before you run superset db upgrade.
Verified metadata database backup
Take a full backup of the metadata database immediately before the window, and confirm it restores. The upgrade guide asks you to back up first, but a backup you have never restored is only a file. Record the backup time, its size and the exact Superset version it was taken from, so you know which release it matches if you need it.
Export of rows the 6.1.0 migration deletes
The 6.1.0 composite primary key migration removes duplicate and NULL rows from many-to-many association tables. If your auditors need a record of those rows, export them before the migration runs. Check UPDATING.md for the affected tables and confirm with your compliance owner whether the export is required.
Inventory of schema-dependent customizations
List every custom configuration key, custom plugin, database driver and script that reads or writes the metadata database directly. Anything that queries Superset tables by hand may break when columns or keys change. Review each item against the release's breaking change notes and decide whether it needs an update before or after the window.
Staging copy restored from production
Restore a recent production backup into a staging metadata database and run the full migration there first. Use the same database engine and version as production, and the same Superset image. Time the run on this copy, because the duration on a large metadata database is the figure your downtime window depends on.
Database engine version check
Confirm the PostgreSQL or MySQL version of the metadata database is in the range the configuration documentation lists. The documentation lists PostgreSQL 10 to 17 and MySQL 5.7 and 8.x. If you run an older version, plan the database upgrade as a separate change with its own backup and test, before the Superset migration.
Named rollback owner with restore access
Assign one person who can make the rollback decision during the window and who holds the credentials to restore the backup and redeploy the previous image. Agree in advance on the failure signal that triggers rollback and the cut-off time after which you stop trying to fix forward. Without this owner, a stalled migration can sit half-finished for hours.
How to run the migration sequence with checks and rollback points
This sequence moves a production Superset metadata database to a new release in a controlled order, with a check at every step and a named rollback point before any schema change runs. Each check confirms that the step finished cleanly before the next one starts.
Freeze configuration and announce the window
The platform owner freezes changes to superset_config.py, the image tag and the database connection settings, then announces the maintenance window to dashboard owners, report recipients and embedded-product teams. Freeze the release you are moving from as well, so staging and production run the same target build. Done means a written change record exists that names the source version, the target version, the config commit hash and the people who can halt the change. Confirm the target version against the UPDATING.md notes for each release between your current version and the target before the window opens.
Stop Celery workers and beat
The operator stops Celery workers and Celery beat so that no scheduled reports, alerts or async queries write to the metadata database during the migration. Check the broker queue and confirm no tasks are mid-flight, then record the queue depth as a baseline. Done means worker and beat processes report stopped, and the queue is either drained or its remaining tasks are recorded for replay after the upgrade. Stopping workers first prevents half-written rows from landing in tables the migration is changing.
Take and verify the metadata backup
The database administrator takes a full logical backup of the metadata database, using the database's native dump tool, and stores it outside the host running Superset. Verify the backup by restoring it into a scratch database, running a row count on key tables such as ab_user, dashboards, slices and the association tables, and comparing the counts to production. Done means the restore test succeeds and the counts match. This verified backup is the rollback point: once the migration starts, restoring it is the only way back for schema changes that cannot be reversed in place.
Export audit rows before the composite key migration
If your target release includes the 6.1.0 composite primary key migration, the security or data owner exports the many-to-many association rows that the migration deletes as duplicates or NULLs. Compare the row counts in the export with the counts in the verified backup and record any differences. Done means an export file with a row count and a checksum is stored with the change record. Skip this step only when your audit requirements confirm that the deleted rows have no retention value.
Rehearse the migration in staging and time it
The team restores the verified production backup into a staging environment on the target release, runs superset db upgrade followed by superset init, and records the elapsed time for each command. Check the migration log for errors and warnings, then run the smoke tests listed in the final step against staging. Done means both commands exit without error, the timing is recorded, and staging dashboards load. Use the recorded duration to set the production window, and ask your team to confirm any table lock behavior called out in the release notes for your database engine.
Run superset db upgrade in production
The operator runs superset db upgrade against the production metadata database from a single host, with the target image or package version already installed and the production config frozen. Watch the output for failed migration revisions and keep the session open until the command returns. Done means the command exits with status zero and the migration log ends without errors. If it fails, stop, do not run superset init, and restore the verified backup to return to the rollback point.
Run superset init and restart web and workers
The operator runs superset init to sync roles and permissions, then restarts the web servers first and the Celery workers and beat second. Confirm that the web tier answers health checks and that the login page loads. Done means superset init exits cleanly, every web instance reports the new version, and workers reconnect to the broker without errors. Restarting web before workers keeps the queue from being consumed by code that expects the old schema.
Run smoke tests on dashboards, charts and reports
The release owner runs a fixed smoke test list: sign in through the configured identity provider, open a representative dashboard from each group, run a chart against each main database connection, trigger an async query, send a test alert or report, and confirm a thumbnail renders. Record pass or fail for each item against the staging results. Done means every item passes and the results are attached to the change record. Keep the window open until the team agrees that the results match staging, and only then announce completion to stakeholders.
Rollback options when a migration fails
A migration that stops partway through leaves the metadata database in an unknown state. Some schema changes may have committed and others not, depending on the database engine and how the migration was written. Do not restart the Superset web or worker processes against the database until you know which state you are in. Start by capturing the exact error and the revision the migration reached, then compare the schema against what the failing step expected.
- Stop further attempts: Halt any scripted or repeated
superset db upgraderuns and keep application pods or processes stopped or on the previous release. - Record the revision: Note the last applied migration revision from the Alembic version table and the error text before changing anything.
- Decide between repair and restore: If the failure is a clear, isolated problem such as a lock timeout or a data conflict that you can resolve, re-running the migration may be appropriate. If the schema is half-changed and you cannot confirm its state, restoring the pre-upgrade backup is the safer route.
- Restore to a separate target first: Where possible, restore the backup into a scratch instance to confirm it is complete before overwriting the production metadata database.
- Assign the decision: The database owner and the Superset platform owner should agree on the path in writing, with the security or compliance lead informed if the rollback touches audit data.
Confirm with your database team how your engine handles DDL inside transactions, because that determines how cleanly a partial failure can be unwound.
Sometimes the migration completes without error, yet dashboards, charts, permissions or reports behave differently after the upgrade. In this case the database is not corrupted, so restoring a backup is rarely the first step. The usual fix is to identify whether the problem is in the application release, its configuration or its data, and roll back the application image or package while keeping the migrated schema only if the older release can run against it. Check the release notes and UPDATING.md for the versions you are moving between before deciding.
- Check version compatibility: Confirm whether the previous Superset release can read the migrated schema. In many cases it cannot, so an application rollback alone may not be possible.
- Compare configuration: Diff
superset_config.pyand environment variables against the last known good release, since removed or renamed settings can cause odd behavior without a failed migration. - Run `superset init` again if needed: Re-running the initialization step can refresh roles and permissions after an upgrade, but confirm its effect in staging first.
- Reproduce in staging: Recreate the issue on a copy of the migrated database to separate application faults from data faults.
- Restore only when data is wrong: If the migrated data itself is incorrect, a restore to a point before the upgrade may be needed, which means losing any changes made since then.
Confirm with the Superset platform owner whether the older release is still deployable from your image registry before the upgrade window begins.
Some migrations in recent Superset lines remove rows rather than only changing structure. The 6.1.0 composite primary key migration, for example, deletes duplicate and NULL rows in many-to-many association tables, according to the UPDATING notes. If you need to keep those rows for an audit trail, you must export them before the migration runs. After the fact, recovery depends on whether you have a usable backup or archived log history, so the options narrow quickly.
- Identify affected tables: Use the UPDATING notes for the target version to list the association tables the migration touches, and compare row counts before and after.
- Use the pre-upgrade export: If the team exported affected rows beforehand, reconcile them with the migrated tables rather than reinserting blindly.
- Use a full backup as a reference: Restore the backup to a separate database and extract the missing rows, then decide how to merge them.
- Consider point-in-time recovery: PostgreSQL can restore to a moment before the deletion, but only if continuous archiving and base backups were configured in advance. Confirm restore timing and the retention of archived logs in your own environment.
- Record the decision: Document who approved the recovery and what was restored, since auditors may ask how the gap was handled.
Confirm with your database owner whether continuous archiving is active today, because it cannot be added retroactively to cover a past migration.
What makes a metadata migration take longer
This table compares the factors that stretch a Superset metadata migration and shows what each one changes in the work. Read each row as a question to answer in your own environment before setting a downtime window. The main takeaway is that database size is only one driver, and the migration plan depends just as much on what sits around the metadata database.
| Driver | Effect on the work | What to ask for |
|---|---|---|
| Metadata database size and table row counts | Larger association and history tables take longer to migrate and back up, and the time needed is hard to predict without measuring a staging copy. | A row count per table from a recent production copy, and a timed migration run on staging before the window is set. |
| Dashboards, charts and users affected by deleted association rows | The 6.1.0 composite primary key migration deletes duplicate and NULL rows in many-to-many association tables, so more affected ownership or relationship data means more to review. | A query-based count of affected association rows, an export of those rows if an audit trail is needed, and a list of owners to confirm afterward. |
| Custom plugins or Python dependencies | Custom code can fail after the schema changes or require a rebuilt image, which adds test cycles around the migration rather than inside it. | An inventory of plugins and pinned packages, and confirmation from the vendor or plugin author that they support the target release. |
| Replication or standby setup | Replicas and standbys must be handled so they do not diverge from the migrated primary, and a failover mid-migration needs a defined response. | The replication topology, the steps to pause or rebuild standbys, and whether a logical replication approach is possible for the database engine in use. |
| MySQL versus PostgreSQL behavior | Each engine locks and rewrites tables differently during schema changes, so the same migration can block writes for different lengths of time on each. | Engine and version as deployed, the lock behavior documented for that version, and a staging test on the same engine and version as production. |
| Backup tooling and restore testing | A backup that has never been restored is not proof of recovery, and restore time sets the real rollback window if a migration fails. | A backup method that produces a consistent snapshot, a timed restore to a separate instance, and a documented point-in-time recovery option where the engine supports it. |
| Number of environments to rehearse | Each environment that has a different data set, version or configuration needs its own rehearsal, which multiplies testing time before production. | A list of environments in scope, the order they will be migrated, and the evidence each one must produce before the next starts. |
Questions teams ask before running Superset migrations
How much does a Superset metadata migration cost?
The cost depends mostly on the team time and environment work around the migration, not the superset db upgrade command itself, which is free to run. The main components are planning and reading the release notes, building a staging copy of the metadata database, rehearsing the upgrade, the backup and restore setup, the cutover window, and post-upgrade testing of dashboards, charts and scheduled reports. Each of these can be done by your team or by a partner under a written scope. Ask any quote to list each component separately, state which items are one-time and which recur (such as backup verification or the next upgrade), and name what drives the effort up, including database size, the number of breaking changes in UPDATING.md and how many integrations depend on Superset. No public figure is given here, so request a quote built from your own scope.
How long does the downtime window need to be?
The window is set by how long the migration takes on your data, which you can only measure in a rehearsal. Migration time grows with the size of the tables that change, whether a step rewrites a table, and whether the database is MySQL or PostgreSQL. Some migrations delete rows or change keys on association tables, so the rehearsal must run against a recent copy of production. Measure three things: the duration of superset db upgrade, the time for superset init, and the time to validate the restored environment. Add time for the backup and a rollback allowance. Confirm the exact steps for your target release in the upgrade guide and UPDATING.md before setting the window.
What does the work need from our team?
Plan for four groups of people. A database owner must confirm backup, restore and privileges on the metadata database. A platform engineer must own the deployment, the configuration, including SECRET_KEY, and the container or package versions. A Superset administrator must test dashboards, charts, datasets, alerts and reports in staging. A change approver must agree on the cutover window, the rollback trigger and who communicates with users. Name these owners before the rehearsal starts, because most delays come from access requests and unclear sign-off rather than from the migration commands.
Can our own team run the metadata migration?
Yes, if your team has administered the metadata database, can restore a backup into staging, and has read the upgrade guide and UPDATING.md for the target release. Teams with these skills often handle routine upgrades alone and bring in a partner for the first upgrade across a major version, for a database that has grown large, or where a migration removes data that auditors may ask about. A partner can also run a fixed-scope rehearsal and write the runbook. Whichever path you choose, the output should be a tested runbook, a verified backup and a documented rollback path.
How do we roll back if a migration fails?
Rollback works only if you restore the metadata database from a backup taken immediately before the upgrade and redeploy the previous Superset version that matches it. Running the newer application against an older schema, or the older application against a newer one, is the main cause of failed rollbacks. Before the window, confirm that the backup restores in a test environment, record the previous image or package version, and store the SECRET_KEY value so sessions and encrypted fields still work. Decide in advance who can call a rollback and at what point. Some migrations delete duplicate or NULL rows, so if you need an audit trail, export the affected association-table rows before running the upgrade.
Should we upgrade PostgreSQL at the same time as Superset?
Usually not. Separating the two changes keeps each failure easy to isolate. Check first whether your PostgreSQL major version is still supported, using the PostgreSQL versioning policy, and confirm that your Superset release supports that version in its configuration documentation. If the database must move, plan the PostgreSQL upgrade as its own change with a rehearsal, and complete it and verify it before the Superset migration, or after a stable period. PostgreSQL supports logical replication between major versions, which can shorten downtime, but it adds components that need their own testing and rollback.
When to coordinate Superset migrations with PostgreSQL upgrades
Do not combine a Superset upgrade and a PostgreSQL major upgrade in one maintenance window unless you have rehearsed each change on its own. When both move at once, a failed superset db upgrade leaves you unsure whether the problem is the Superset migration, the new database engine, or the data transfer between them. Confirm the PostgreSQL version your metadata database runs against the versions Superset documents as supported, which the configuration documentation lists as PostgreSQL 10 to 17. Then check the PostgreSQL versioning policy, since PostgreSQL 14 is listed as supported only until November 12, 2026, and plan the database upgrade on its own schedule if that date falls near your Superset work.
PostgreSQL is released under the PostgreSQL License, a permissive license similar to BSD and MIT, and each major version receives support for five years. If you must change the major version, the PostgreSQL upgrading documentation describes three routes: a dump and restore, *pg_upgrade*, and *logical replication*. Logical replication works across major versions and can keep downtime to several seconds, but it needs its own rehearsal, replication monitoring and a cutover plan that points Superset at the new primary. Your DBA team should confirm which route fits the database size and the restore time they have measured in staging.
PostgreSQL does not detect a failed primary or promote a standby by itself, so automatic failover depends on external tools that your platform team must select, configure and test. Make sure the Superset SQLALCHEMY_DATABASE_URI target and any connection pooler, such as PgBouncer, will follow a cutover, and decide whether the migration runs before or after that cutover. Record the order of operations in the runbook so the on-call engineer can tell which system changed if the window overruns.
How DeployOpen plans and runs metadata migrations
DeployOpen treats the Superset metadata database as the asset most worth protecting during an upgrade. We scope the migration against your database size, recovery objectives and access model, then rehearse each step in staging before any production change.
How we scope
- Inventory the metadata database version, size, extensions and any custom tables that the migration could touch.
- Review UPDATING.md for every release between your current version and the target, and list breaking changes that affect your deployment.
- Agree in writing whether an audit trail is needed for association-table rows that the 6.1.0 composite key migration removes.
- Define the downtime window, rollback point and sign-off owners before work starts, since engagements are fixed-scope.
How we build
- Build a staging copy of the metadata database and run superset db upgrade and superset init against it first.
- Time each migration step on the staging copy and flag any statement that takes exclusive table locks in production.
- Export affected association-table rows before the migration where the scope requires an audit record, and verify the export.
- Write and test the rollback path, including restoring the database backup and reverting the application image.
How we hand over
- Deliver a written runbook covering backup, migration order, verification queries, downtime steps and rollback criteria.
- Document the exact Superset and database versions tested, and the staging results your team can compare against production.
- Walk your operators through the cutover and post-upgrade checks, including dashboard, alert and report smoke tests.
- Record the configuration changes made during the upgrade so the next migration starts from a known baseline.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers for Superset and its metadata database.
- Retainer support can cover planning the next Superset release, rehearsing its migrations and reviewing database maintenance.
- Confirm with your database team how backups, point-in-time recovery and PostgreSQL or MySQL version support are owned after handover.
- Agree how migration issues are escalated and which upstream release notes or project channels your team monitors.
