Why Apache Superset high availability needs more than extra web nodes
Once Superset becomes the layer your teams use to read the numbers, an outage does more than close a browser tab. Alerts stop firing, scheduled reports stop arriving and async queries hang. High availability for Superset means more than running several web nodes. It means redundant web nodes behind a load balancer, a durable metadata database with tested backups, a redundant caching layer, redundant Celery workers and beat processes, and a recovery procedure your team has actually rehearsed. Each of these pieces can fail on its own, and a design that protects only the web tier leaves the features that matter most to the business exposed.
The harder part is that the web tier is the easy part. The Superset architecture documentation states that alerts and reports, async queries and dashboard thumbnails depend on the caching layer and the worker components, and most installations use Redis and Celery for them. If Redis or the workers are down, the dashboards may still load while reports silently stop. Your monitoring and runbooks should therefore check queue depth, worker health and cache availability as first-class signals, not only HTTP responses from the web nodes. Ask the team who will own that monitoring and who gets paged when a scheduled report fails to run.
Some defaults that work for a trial are not production choices. The architecture documentation says the SQLite default used by Quickstart and PyPI installs is not recommended for production, and the same applies to the PostgreSQL container in the Docker Compose setup. Superset is tested with PostgreSQL and MySQL as the metadata database, and the configuration documentation lists supported versions of each. Confirm the exact versions you plan to run against the current documentation before you commit to a database platform, and decide early whether the database is a managed service or a self-run cluster, since that choice drives backup, failover and patching responsibilities.
Every web node and worker must share the same SECRET_KEY. Superset requires a user-specified key to start, and the configuration documentation describes rotation with PREVIOUS_SECRET_KEY. If nodes hold different keys, sessions and encrypted values written by one node will fail on another, and users will see intermittent errors that look like random outages. Store the key in your secrets manager, inject it identically into every container or virtual machine, and include the rotation procedure in your recovery tests so that a failover does not depend on one person's copy of the key.
The Apache-2.0 license places no paid edition between you and the software, so the design, operation and testing of this highly available setup belong to your team or to the partner you hire. Plan for the work to recur: upgrades, backup restore drills and failover tests all take time after go-live. Before you sign a scope, confirm who writes the runbooks, who runs the failover test, how long a restore is expected to take in your environment, and what evidence will be kept for your change and audit records.
Which Superset components need redundancy and why
Superset is split into four architecture components, and they do not fail in the same way. Some hold state and some do not, and the optional layers matter most for alerts, reports, async queries and dashboard thumbnails. Review each component against the question of what stops working when it goes down.
Flask and React application
This is the web application that serves the user interface and API, built on Flask and React. It is the component most teams scale horizontally, because the application tier is designed to run as several nodes behind a load balancer. Confirm with your own configuration that every node uses the same SECRET_KEY and points at the same metadata database, or sessions and logins will behave inconsistently between nodes. If a single node fails, users on that node lose their session, but the other nodes keep serving requests.
Metadata database
The metadata database stores Superset's own state: dashboards, charts, datasets, users, roles and permissions. It is the only component that holds the definitions your users depend on, so losing it means losing the work that lives there, not just uptime. Superset is tested with PostgreSQL and MySQL, and the SQLite default and the Docker Compose PostgreSQL container are not recommended for production. Plan a managed or clustered database with tested backups and a restore procedure, and confirm the supported versions in the configuration documentation before you choose one.
Optional caching layer
The caching layer holds temporary data that speeds up queries and supports background features. The architecture documentation says alerts and reports, async queries and dashboard thumbnails require this layer, and most installations use Redis for it. If the cache fails, dashboards may load more slowly, and features that depend on it stop working, but the metadata database remains intact. Ask your team whether any of your required features depend on it and decide whether it needs its own replication or failover design.
Optional Celery worker and beat processes
Celery workers run the asynchronous jobs: scheduled alerts and reports, async queries and thumbnail generation. Celery beat issues the scheduled tasks, so a beat process that is not running means schedules silently stop firing. Most installations use Celery with the caching layer as its broker. Run more than one worker so a single failure does not stop all background jobs, and monitor beat separately, because one beat process is the usual design and a missed schedule is easy to overlook.
Decisions to settle before you add a second node
This table compares the infrastructure choices to settle before adding a second Superset web node, with one row per option. Read each row for when the option fits and what it adds to daily operations, since costs depend on your own staffing and environment. The main takeaway is that the metadata database and cache choices set your failover path, so settle them before you size anything else.
| Decision and option | When it fits | What it costs operationally |
|---|---|---|
| Managed PostgreSQL or MySQL metadata database | Teams without a database team that want the provider to handle patching and failover. Check the version against PostgreSQL 10 to 17 or MySQL 5.7 or 8.x. | Less routine work, but you still own upgrade migrations and restore testing. Confirm failover and backup retention in the provider's terms. |
| Self-managed metadata database | Organizations with an established database platform, standard backup tooling and staff who run replication and failover. | Full control over replication and failover, with patching, backups and on-call effort carried by your team. |
| Redis as a single managed instance | Early or low-load deployments that can tolerate a cache and broker restart while service is restored. | Lowest setup effort, but a failure interrupts async queries, reports and thumbnails until the instance recovers. |
| Redis clustered or replicated | Deployments where scheduled reports and async queries must keep running through a node failure. | Adds topology, failover and client configuration work. Confirm the topology is supported in the configuration docs for your version. |
| Co-located Celery workers | Small installations with light background load, where one host runs both the web application and workers. | Simpler to run, but a worker spike competes with web requests for memory and CPU on the same host. |
| Separate Celery worker pools | Installations where reports, alerts or long queries must not slow dashboard loading. | More hosts and scaling rules to manage, plus separate monitoring for each pool and the beat process. |
| Cross-zone placement within one region | Services where a single zone outage must not stop dashboards, and the database, Redis and web nodes can span zones. | More placement rules and cross-zone dependencies to design and test. Confirm zone-level failover for each managed service. |
| Virtual machines | Teams with existing VM automation, load balancers and patch processes, and no Kubernetes platform team. | Familiar tooling, but you script node replacement, configuration drift control and rolling upgrades yourself. |
| Kubernetes with the Superset operator | Teams already running Kubernetes with a platform team that accepts alpha-level change risk. | The operator is alpha, with a v1alpha1 API that may change between releases. Plan extra upgrade testing. |
How to build a highly available Superset in eight steps
This sequence moves Apache Superset from a single installation to a deployment where any web node can serve any user, background work continues if one component restarts, and the team has tested recovery before go-live. Each step ends with a check you can record.
Provision an external metadata database with backups
The metadata database stores dashboards, users, roles and settings, so it is the first dependency to make highly available. Use PostgreSQL or MySQL within the supported versions listed in the configuration documentation, which are PostgreSQL 10 to 17 and MySQL 5.7 and 8.x. Keep the SQLite default and the Docker Compose PostgreSQL container out of production. Your DBA or platform team configures backups and point-in-time recovery. The step is done when a backup restores to a separate instance and Superset starts against the restored copy.
Set one shared SECRET_KEY and plan rotation
Superset requires a user-specified SECRET_KEY to start, a requirement added in version 2.1.0. Every web node and worker must read the same value, so store it in your secrets manager and inject it identically into each container or host. Before go-live, document the rotation path: set the new key as SECRET_KEY, place the old value in PREVIOUS_SECRET_KEY, roll out the change, and remove the previous value once you have confirmed that sessions and encrypted fields still work. The check is that a session created on one node is accepted by another.
Run identical stateless web nodes behind a load balancer
Deploy at least two web nodes from the same image and the same configuration. Keep local state off the nodes, including uploaded files and any local cache that users depend on, so that a node can be replaced at any time. Configure the load balancer health check against the health endpoint your Superset version exposes, and confirm the path in its documentation. The check is a controlled test: stop one node, confirm the balancer removes it from rotation, and confirm dashboards keep loading through the remaining node.
Move cache and result backend to redundant Redis
The architecture documentation states that the caching layer supports alerts and reports, async queries and dashboard thumbnails, and most installations use Redis for this role. Deploy Redis with replication and automatic failover in your environment, then point the cache and result backend settings at the shared endpoint. Confirm that the web nodes hold no cache data locally. The check is a failover test on the Redis tier: after a primary is replaced, Superset reconnects without a restart of the web nodes, and the team records how long reconnection takes.
Run Celery workers and beat as separate processes
Run Celery workers and the beat scheduler as their own processes or containers, separate from the web nodes, so that each can restart without affecting the user interface. Beat should run as a single scheduler instance, which your deployment design must enforce. Workers connect to the same broker and the same metadata database as the web tier. The check is a restart test: restart one worker while async queries are queued and confirm they finish on another worker, then restart beat and confirm scheduled jobs resume without duplicate runs.
Confirm alerts, reports and async queries from every node
Alerts, reports, async queries and thumbnails depend on the worker and cache components, so test each one with the web node that serves the request and the worker running on a different host. Start an async SQL Lab query on one web node, confirm a worker completes it, and confirm the results are readable from another web node. Send a test report and a test alert through the scheduler, then check delivery and confirm the schedule fires once. The check is a written test log for each feature, signed off by the team that owns the deployment.
Put session and identity settings in shared configuration
Any node should serve any user, so session and identity settings must be identical on every node. This covers OAuth2 or LDAP settings, the AUTH_ROLES_MAPPING configuration that maps groups to roles, and the session backend. Deliver the configuration file from one source, such as a version-controlled repository or a configuration secret. Register the OAuth callback URL against the load balancer's public hostname. The check is a cross-node login test: sign in through one node, move to another node without signing in again, and confirm the same roles apply on both.
Document the runbook and test recovery in staging
Write a runbook that covers node replacement, Redis and database failover, secret rotation, worker and beat restarts, and restore from backup. For any change, back up the metadata database first, then follow the upgrade order in the upgrade documentation: run superset db upgrade and superset init, after reviewing UPDATING.md for breaking changes between versions. Test every change in staging first. The check is that the team performs one restore and one rolling node replacement in staging, then updates the runbook with anything the test showed was missing.
What drives the effort and cost of a highly available Superset
This table lists the factors that change the size of a highly available Superset rollout and shows how to keep each one under control. Read each row as a question to answer for your own environment before asking for a fixed-scope quote. The main takeaway is that scheduled reports, recovery targets and identity complexity usually move effort more than raw user counts do.
| Driver | Why it changes effort or cost | How to keep it down |
|---|---|---|
| Concurrent users and dashboards | More concurrent queries need more web nodes, cache capacity and tuning of the metadata database connection pool. | Start with a small number of web nodes behind the load balancer and add nodes only when monitoring shows load. Move heavy dashboards to cached or pre-aggregated datasets. |
| Scheduled reports and alerts | Each report or alert runs through Celery workers and depends on the caching layer and thumbnail rendering, so job volume drives worker sizing. | Group scheduled jobs into separate worker pools by priority, stagger schedules and retire unused reports before go-live. |
| Connected data sources | Each database needs its own connection setup, credentials, network path and testing, and some sources need extra drivers in the image. | Limit the first release to the sources that feed approved dashboards and add others in later, scoped change requests. |
| Recovery time and recovery point targets | Shorter targets require standby database replicas, tested restore procedures and automated failover for the cache and workers. | Agree written targets with the business owner early. Use a managed database service with backups where your platform team already runs one. |
| Number of environments | Staging, production and disaster recovery each need the full stack, data refresh and access controls, and upgrades must be rehearsed in staging first. | Keep one staging copy for upgrade testing and build production and recovery from the same infrastructure templates to avoid drift. |
| Identity integration complexity | OAuth2, LDAP or remote user headers each need their own configuration, group mapping through AUTH_ROLES_MAPPING and testing of role changes. | Pick one identity protocol for the first release, map groups to a short list of roles and document the mapping in a single file under version control. |
| Network and compliance constraints | Private networks, restricted egress, approved images, key management and evidence collection add design and review time. | Confirm network zones, image sources and secret storage with your security team before design starts. Reuse controls already documented for other internal platforms. |
| Customization of configuration or security manager | Custom configuration overrides and a custom security manager must be retested after every Superset upgrade and can block routine version changes. | Keep overrides in a small, reviewed configuration module. Avoid custom security manager code unless required, and check UPDATING.md before each upgrade. |
How the design changes for single node, virtual machines, Kubernetes and air-gapped sites
A single node runs the application, metadata database, cache and workers on one host. It is the simplest layout to install and the least forgiving when something fails. It suits evaluation, a proof of concept or a small test environment, but it does not protect against host hardware loss, disk failure or an upgrade that leaves the only web process down.
- Protects against: process failures, if a process supervisor restarts the services (confirm the setup on your host).
- Does not protect against: loss of the host, its disk, a data center outage or a failed in-place upgrade.
- Metadata database: the SQLite default of Quickstart and PyPI installs is not a production option, and neither is the PostgreSQL container from the Docker Compose setup. Use PostgreSQL 10 to 17 or MySQL 5.7 or 8.x, which the configuration documentation lists as supported, on infrastructure you back up separately.
- Cache and workers: alerts, reports, async queries and dashboard thumbnails depend on the caching layer and Celery workers, so they fail together with the host.
Confirm the recovery time your team would accept if the host were lost, and whether someone can rebuild the node from backups within that window.
Virtual machines behind a load balancer are the most direct pattern for stateless web nodes. Each VM runs the same Superset application and configuration, the load balancer spreads requests across them, and shared state lives in the external metadata database, cache and workers. The main design work is making sure every node behaves identically, so a session or background result is handled the same way wherever a request lands.
- Shared secret: every node must use the same SECRET_KEY. Superset requires one to start, and PREVIOUS_SECRET_KEY supports rotation during a planned window.
- Identical configuration: deploy the same configuration files and overrides to every node from one source of truth, so no node drifts.
- Health checks: point the load balancer at a health endpoint on each node so a failed node leaves rotation (confirm the endpoint for your version).
- Separate workers: run Celery workers and beat on their own hosts, so losing one web VM does not stop scheduled reports.
- Rolling upgrades: drain one node, upgrade it, test, then return it to service. Run database migrations once for the release, not on each node, and read UPDATING.md before starting.
Confirm how your load balancer handles session affinity and whether your authentication setup depends on it.
On Kubernetes, the cluster handles scheduling, restarts and scaling of web pods, but it does not hold the metadata database, cache or worker state for you. The deployment choice matters more than usual here. The official Helm chart is documented for the 6.1.0 release, while the current development documentation marks that chart as deprecated and points new deployments to the Apache Superset Kubernetes Operator.
- Helm chart: documented for 6.1.0 and marked deprecated in the current development docs, so check whether it fits your timeline before you build around it.
- Operator: built with the Go Operator SDK and in alpha, with a v1alpha1 API whose custom resource schemas may change between releases.
- Upgrade effort: expect manifest changes when the operator API changes, so pin versions and budget time for reviewing them.
- External state: keep the metadata database, cache and workers as managed or separately run services with their own backups and recovery tests.
- Pod settings: set the SECRET_KEY through a Kubernetes secret, define readiness and liveness probes, and run more than one web replica.
The operator or chart decision affects the build sequence and the upgrade plan, so settle it first. See the sibling topic on choosing between the operator and the Helm chart.
An air-gapped site cannot reach public package indexes or container registries during installation or upgrades, so every dependency must be staged inside the network before work begins. Availability at install time is the main design constraint. Patches and security fixes must also cross the air gap on a schedule your team controls.
- Python packages: mirror the exact Superset release and its full dependency tree in an internal package index, since reaching PyPI at install time is no longer possible.
- Container images: mirror the Superset image and every image it needs, including the cache, database and any chart or operator images, into an internal registry.
- Data source drivers: stage the database drivers for each analytics source, and confirm any system libraries they require are included.
- External calls: test each feature that fetches external resources, such as thumbnails, alerts or email, and confirm it works offline or is disabled deliberately.
- Upgrade path: keep release notes and UPDATING.md with each mirrored version, so transfers match what you test.
Build the environment from a clean offline segment and confirm every dependency resolves before go-live, because a missing package is usually discovered during the outage it was meant to prevent.
Setting recovery time and recovery point targets for Superset
Set recovery time objective (RTO) and recovery point objective (RPO) targets for the three parts of a Superset deployment separately: the dashboard layer, the metadata database and the scheduled jobs. A single number for the whole platform hides the fact that these parts fail and recover in different ways. The web nodes are stateless, so losing one is usually a capacity event that a load balancer and a replacement node absorb. The metadata database is the stateful core, and the scheduled jobs depend on both the database and the workers. Each needs its own answer to two questions: how long can it be unavailable, and how much recent change can it lose.
The metadata database deserves the most attention because it defines what a restore means. It holds dashboards, charts, datasets, users, roles and the permission model that your access controls rely on. If you restore the database to a point in time, you restore those definitions to that point, and any change made after it is gone. Users who were added or re-mapped after that point, through the group mapping configured with AUTH_ROLES_MAPPING or through manual edits, may need to be re-provisioned. Set the database RPO with that in mind, and decide whether dashboard edits made by analysts count as data you must not lose.
Scheduled jobs such as alerts, reports and dashboard thumbnails rely on the caching layer and Celery workers, and they read state from the metadata database. Their RTO is usually looser than the dashboard layer, but a missed report can still matter to a business owner. Decide whether a missed scheduled run is acceptable to be re-run after recovery or must be delivered within a defined window. The caching layer is generally rebuildable, so its RPO can be effectively reset on restore, but confirm that with your own cache configuration and any queued task state you depend on.
Targets must be agreed with the business owners of each dashboard group, not chosen by the platform team alone. Ask owners which reports are used for daily decisions, which are regulatory or customer-facing, and how long the business can run without them. Record the agreed values for each component, the owner who accepted them, and the date, and write them into the runbook alongside the restore steps they depend on. A target that no one has accepted will not guide decisions during an incident.
Before you commit to any figure, confirm backup frequency, point-in-time recovery support, backup retention and expected restore duration with your database provider, or with your own team if you run PostgreSQL or MySQL yourself. Those values depend on your provider, your plan and your environment, and this page does not state them. Then test a restore into a non-production environment and measure how long it takes. The measured time, not the vendor's general description, is the number to compare with your agreed RTO.
Protecting secrets, sessions and access across redundant nodes
Every Superset web node in a highly available setup must agree on secrets, sessions, identity and transport settings, or users will see intermittent failures and security gaps will appear on some nodes but not others. The cards below cover the points that change when you run more than one node, and they point to the hardening and SSO topics for full detail.
Identical SECRET_KEY on every node
Superset requires a user-specified SECRET_KEY to start, and every web node must use the same value. If nodes disagree, sessions signed by one node fail on another, so users appear to be logged out at random. Store the key in your secrets manager and inject it at startup, never in a configuration file committed to source control. Confirm that every node and every Celery worker reads the same value after each deployment.
Rotating keys with PREVIOUS_SECRET_KEY
Superset supports rotating the SECRET_KEY through PREVIOUS_SECRET_KEY, which lets the platform accept sessions signed with the old value while the new value takes effect. Plan rotation as a staged change across the node pool rather than a single restart, and record which value each node holds during the window. Ask your implementation partner to document the exact rotation sequence for your deployment and to test it in staging before production.
Database and Redis credentials
The metadata database and the Redis cache or Celery broker hold configuration, query history and task state, so their credentials need the same care as the secret key. Restrict network access so that only the Superset web nodes, workers and beat processes can reach these services, and rotate their passwords on a schedule your security team sets. Verify rotation with a test deployment, because each node holds its own connection string until it restarts.
TLS at the load balancer and inside the cluster
Terminate TLS at the load balancer with certificates managed centrally, then decide whether traffic between the load balancer and Superset nodes, and between nodes and the database or Redis, must also be encrypted. Internal encryption adds certificate management work on every node, so the choice should follow your network policy. Confirm which hop carries HTTPS, how the application learns the original scheme, and that session cookies are marked secure.
Identity provider settings matched on every node
OAuth2 or LDAP settings, including client IDs, callback URLs and AUTH_ROLES_MAPPING, must be identical on all nodes, otherwise a login may succeed on one node and assign different roles on another. Keep the configuration in one versioned source and deploy it to every node together. The SSO and identity integration topic covers provider setup and group mapping in depth.
Hardening and security review references
Least-privilege roles, row-level security, data source credential handling, SQL Lab exposure and published vulnerability tracking all apply to a redundant deployment in the same way they apply to a single node. Use the security hardening topic as the checklist for those controls, and confirm that each node runs the same patched release. This topic covers only the points that change because the platform runs on several nodes.
How to test that Superset fails over and recovers
A high availability design for Superset is only proven when each failure has been induced on purpose and the result written down. The test plan below covers the components that matter most: the web nodes behind the load balancer, the Celery workers that run alerts, reports and async work, the Redis cache, and the metadata database. Start by agreeing the list of tests with the team that will run the platform, because each test needs a named owner, a staging environment that mirrors production topology, and a window in which load can be generated safely.
Web node loss under load. Generate sustained dashboard traffic through the load balancer, then stop one Superset web node. Confirm that existing sessions keep working after a retry, that dashboards render, and that the load balancer stops routing to the failed node within the health check interval you configured. Check that session state is not stored only on the node that failed. If users are forced to sign in again, record that as the expected result only if your identity integration was designed for it. Repeat with the node restored to confirm it rejoins the pool without manual changes.
Celery worker loss and queue continuity. Schedule a test alert and a test report so that one is due while a worker is stopped. Stop the worker that picks up the task and confirm that the queued work runs on another worker, or that it fails with a visible status that someone is alerted to. Also confirm that the beat process is running in exactly one place, since duplicate schedules are a common failure mode. Verify delivery of the report output, not only the task status, because a task can show as complete while sending nothing.
Redis restart and async query behavior. Restart the Redis instance during a set of asynchronous SQL Lab queries and dashboard thumbnail generation. The expected result should be written as a choice: either the queries recover and complete, or they fail with a clear error message that users can understand and retry. Silent hangs are a fail. Check that Superset reconnects to Redis without a web node restart, and confirm that the cache layer recovers to a normal hit rate. Confirm the exact behavior against the version you run, since cache and async behavior changes between releases.
Metadata database restore and SECRET_KEY rotation. Restore the metadata database from the most recent backup into a staging environment, point a staging Superset at it, and confirm that dashboards, datasets, charts, owners and roles load and that a sample user sees the access they should. Then rotate the SECRET_KEY in staging using PREVIOUS_SECRET_KEY as documented, and confirm that existing sessions and encrypted data source credentials still work during the rotation window. Finally, run a timed failover drill that combines a database failover and a web node loss, and record the elapsed time against the recovery objective agreed with the business owner. Every test needs a written expected result before it is run, plus a dated pass or fail record with evidence such as logs, screenshots and timings. Repeat the full set after each upgrade, because a platform change can alter failover behavior even when the architecture has not changed.
What to monitor, back up, test and document after go-live
This table lists what your operations team should watch, back up, test and document once a highly available Superset deployment is live, with the frequency or trigger for each item. Read each row as a recurring control: the first column names the area, the second says what to check or do, and the third says how often or when. The main takeaway is that the metadata database restore test and the Celery and Redis checks matter most, because scheduled reports and async queries fail quietly when those layers degrade.
| Area | What to watch or do | How often or when |
|---|---|---|
| Web node health and load balancer checks | Confirm each Superset web node passes the load balancer health check, and remove any node that fails it from rotation. Watch response times and error rates per node so one unhealthy node does not hide behind the others. | Continuously through alerting, with a manual review at each release |
| Metadata database backups | Confirm automated backups of the PostgreSQL or MySQL metadata database are running, and that retention and encryption match your data policy. Include the database in your change process so backups are taken before any upgrade. | Scheduled backups, plus a backup before every Superset upgrade |
| Metadata database restore tests | Restore a recent backup into a separate environment, run Superset against it, and confirm dashboards, datasets, users and roles load. Record the time the restore took and any manual steps required. | At least on a set schedule agreed with your team, and after any change to the database platform |
| Redis memory and availability | Track Redis memory use, eviction behavior and whether the instance is reachable from web nodes and Celery workers. Confirm the cache layer survives a node restart, since async queries and thumbnails depend on it. | Continuously through alerting, and during each failover test |
| Celery queue depth and worker liveness | Watch the length of the task queues and confirm each worker and the beat process are alive and consuming tasks. A growing queue or a silent worker means reports and async queries are waiting. | Continuously through alerting, with a review after any peak reporting period |
| Scheduled report delivery success | Check that each scheduled alert and report was sent, and investigate failures in the delivery logs. Confirm the SMTP or Slack credentials used for delivery still work. | Daily or per schedule for critical reports, and after any credential change |
| Certificate and secret rotation | Track expiry dates for TLS certificates on the load balancer and web nodes. Rotate SECRET_KEY using PREVIOUS_SECRET_KEY as described in the configuration documentation, and rotate database, cache and data source credentials. | Certificates on a schedule set before expiry, secrets on your security policy cycle and after any suspected exposure |
| Documentation updates | Keep the runbook current with failover steps, restore procedure, on-call contacts and upgrade steps. Update the architecture diagram whenever nodes, databases, cache or workers change, and file the change record. | After each change to the platform, and at least on a regular review date your team sets |
Questions teams ask before running Superset in high availability
How much does Superset high availability cost to run?
The cost is a services fee for planning, building, testing and handover, plus the infrastructure and internal time that keep the platform running.
One-time costs include architecture and recovery design, staging builds, failover tests and cutover.
Recurring costs include the web nodes behind the load balancer, a managed metadata database with backups, cache and worker capacity for Redis and Celery, monitoring, and any support or upgrade retainer you choose.
The main drivers are the number of nodes and environments, how many data sources and identity providers connect, and the recovery targets you set. DeployOpen does not publish a price list. It provides a fixed-scope quote from the written scope, so ask that each line item name the component and the party doing the work.
How long does a Superset high availability project take?
Duration depends on the drivers, so no single timeline applies to every environment. The biggest factors are how quickly your team provisions a managed database and load balancer, how long identity provider configuration and approvals take, the number of data sources to validate, and how many failover scenarios you test.
Ask for a phased plan with a milestone per phase: design and recovery targets, staging build, identity and data source validation, failover and restore tests, then production cutover. Each phase should state its entry criteria and the owner who signs it off. DeployOpen agrees scope, deliverables and timelines in writing before work starts, so the plan should appear in the written proposal.
Can our own team run Superset in high availability alone?
Yes, if your team already has solid skills in database operations, networking and Python operations, and it maintains a clear runbook.
Superset needs a user-specified SECRET_KEY to start, and that key must be consistent across web nodes. Upgrades follow a defined sequence: back up the metadata database, run superset db upgrade and superset init, test in staging, and read UPDATING.md for breaking changes. Your team must also operate Redis, Celery workers and beat, load balancer health checks and restore drills.
Teams with gaps in one area often bring in a partner for design review or failover testing only. A written runbook covering failover, restore, secret rotation and upgrades is the practical test: if an on-call engineer can follow it at 2 a.m., the team can run the platform.
What do we need to provide for a Superset high availability project?
Plan to provide access, information and named people before work starts.
Access: development, staging and production environments, cloud or data center accounts, and visibility of the metadata database, cache and worker hosts.
Identity: your identity provider type (OAuth2 or OIDC, LDAP or remote user headers), client registration details, and the group names you want mapped to Superset roles through AUTH_ROLES_MAPPING.
Data: a list of every data source Superset will query, with owners and how credentials are handled.
Recovery: target recovery time and recovery point objectives for each environment, approved by the business owner.
People: named owners for infrastructure, identity, data and security, plus an agreed change window. Missing items usually appear as schedule risk, so confirm them in the written scope.
Is Apache Superset licensed, or is there a paid edition?
Apache Superset is an Apache Software Foundation project released under the Apache License 2.0 (SPDX identifier Apache-2.0), and the project has no paid edition.
The cost of running it is therefore the infrastructure, people and support around it: web nodes, a managed metadata database, Redis, Celery workers, monitoring, and any services fee for implementation or support.
Check the license terms of the components you add, such as your database service, cache, cloud provider and any managed Superset offering you compare against, since those carry their own terms. Ask your legal or procurement team to confirm the license position for your deployment before you commit.
Which metadata database versions work for a highly available Superset?
Superset is tested with PostgreSQL and MySQL as the metadata database. The configuration documentation lists supported versions as PostgreSQL 10 to 17 and MySQL 5.7 and 8.x.
The SQLite default used by Quickstart and PyPI installs is not recommended for production. The Docker Compose PostgreSQL container is also not recommended for production, according to the architecture documentation.
For high availability, use a managed or replicated database with automated backups, restores you have tested, and failover behavior you have verified. Confirm the version you choose against the documentation for the Superset release you deploy, because supported versions can change between releases.
What breaks first when Superset runs on multiple web nodes?
Most failures come from state that is shared incorrectly across nodes. Check these first:
- SECRET_KEY must match on every web node, or signed data and sessions fail.
- Cache and workers must be reachable from all nodes, because alerts and reports, async queries and dashboard thumbnails depend on the caching layer and worker components.
- Scheduled jobs need a single scheduler, so confirm that only one beat process runs alerts and reports.
- Health checks on the load balancer should test the application, not only the open port.
- Local files on one node should not hold state that other nodes need.
Test each item by stopping one web node, one worker and the primary database during staging tests, then record what users see and how long recovery takes.
Should we run Superset on Kubernetes with Helm or the operator?
Start by deciding how much risk your team can carry. The current development documentation marks the official Helm chart as deprecated and points new deployments to the Apache Superset Kubernetes Operator.
The operator is in alpha, uses a v1alpha1 API, and its custom resource schemas may change between releases. The 6.1.0 documentation still describes the Helm chart, so teams on that release may already run it.
Choose the path based on your risk tolerance, your upgrade cadence and whether you can absorb schema changes during upgrades. If you start with the chart, document a migration plan to the operator. Confirm the operator's current status with the project before you commit to a production design.
How DeployOpen scopes, builds and hands over Superset high availability
DeployOpen plans, builds and documents Apache Superset high availability inside your own infrastructure, starting from a written scope that names every component in the redundant design. Each engagement ends with tested failover, documented runbooks and a clear handover to your team.
How we scope
- Inventory the web, metadata database, cache and Celery worker components your dashboards and alerts depend on
- Agree which node counts, load balancer type and database arrangement are in the written scope
- Confirm whether alerts, reports, async queries and thumbnails must keep working during a node failure
- Record the target environment: cloud account, data center or air-gapped network, and who operates it
How we build
- Run several stateless Superset web nodes behind your load balancer with a shared SECRET_KEY configured identically
- Move the metadata database to a managed PostgreSQL or MySQL service in a supported version, not the SQLite default
- Deploy redundant Redis and Celery workers for alerts, reports and async queries, with configuration kept in version control
- Test node loss, database failover and worker restarts against the agreed scope before go-live
How we hand over
- Document the architecture, configuration overrides and secret rotation steps using the PREVIOUS_SECRET_KEY process
- Provide backup and restore runbooks for the metadata database, with a restore test your team can repeat
- Write the upgrade procedure: back up the metadata database, run database upgrades, test in staging and check UPDATING.md
- Walk your operations team through monitoring, alert thresholds and escalation paths before acceptance
How we support
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts
- Ongoing support, upgrades and recovery testing are available as a retainer if your team wants continued help
- Ask any quote to separate one-time build work from recurring operations, monitoring and upgrade effort
- Confirm with DeployOpen which Helm chart or Kubernetes operator approach the written scope covers before deployment
