Why superset_config.py drifts between environments and how to stop it
Keep one versioned base superset_config.py, inject secrets and per-environment values at runtime, and review every change before it reaches an environment. Teams that run Superset in development, staging and production usually see drift when someone edits a live file, copies a setting into one environment only, or hard-codes a password to get a deployment working. The result is that staging stops predicting production behavior, and nobody can say which settings are authoritative. The Superset architecture documentation lists four components: the Flask and React application, a metadata database, an optional caching layer and optional Celery worker and beat processes. The configuration file is where most of those components get their settings, along with the Flask app config, authentication, and feature flags.
Because the file touches so many parts of the stack, small differences have large effects. A cache or Celery setting that differs between the web tier and the workers can make alerts, reports and async queries behave differently from one environment to the next. Authentication and role mapping settings that exist only in production can make a bug invisible in staging. The SECRET_KEY setting is a useful example: Superset requires a user-specified value to start, and the configuration documentation describes PREVIOUS_SECRET_KEY for rotation. Each environment needs its own key, and the team should confirm how that key is stored, rotated and kept out of Git. Confirm the exact behavior for your version in the Superset configuration documentation.
The project is open source under the Apache License 2.0 and has no paid edition, so no vendor enforces configuration standards for you. Ownership of drift control sits with your platform and security teams. Decide who may change the base file, which values may differ by environment, and how changes are tested in staging before production. Those decisions, and the review process that enforces them, are the work this topic describes in the sections that follow.
Which settings belong in code, environment variables or secrets storage
Structural settings describe how Superset behaves in every environment. These include feature flags, the cache type and the authentication class. They belong in superset_config.py, which is committed to Git, reviewed like any other code and promoted through the same pipeline as the rest of the deployment. Because Superset reads this file as Python, a change to it is a code change and should be treated as one.
- Feature flags: Keep the
FEATURE_FLAGSdictionary in version control so every environment exposes the same features unless a difference is deliberate and documented. - Cache and worker types: Record the chosen cache backend and Celery setup in code so a fresh build reproduces the same behavior.
- Authentication class: Set the security manager class and auth type in the repository, not on individual hosts.
- Imports and helpers: Put any custom logic in a committed module that the config file imports, and keep it free of hard-coded credentials.
- Review trail: Require pull request review for changes so the history explains why a flag or class changed.
Before relying on a key, confirm its name and default against the configuration documentation for your Superset version.
Per-environment values differ between development, staging and production but are not secret. Examples include public hostnames, the database host and port, the number of worker processes and the Redis host. The usual pattern is to read these from environment variables with os.environ in superset_config.py, using a sensible default for local work. This keeps one file for every environment while the deployment supplies the differences.
- Hostnames and public URLs: Read the external address from an environment variable so links and redirects match each environment.
- Database host and port: Point the metadata database at a different host per environment without editing code.
- Worker and cache settings: Vary Celery concurrency and Redis endpoints by environment, sized to the hardware each one runs on.
- Explicit defaults: Define fallback values in code for local development, and make production fail fast when a required variable is missing.
- Environment inventory: Keep a written list of required variables per environment so operators can check a deployment before it starts.
Check that every variable name you use appears in your version's configuration documentation and that your deployment tooling sets it in each environment.
Secrets include the SECRET_KEY, database passwords, OAuth client secrets and, during rotation, PREVIOUS_SECRET_KEY. None of these should be written into superset_config.py or committed to Git. Supply them at runtime from a secrets manager, an orchestrator secret object or a restricted environment file, and read them through os.environ. Superset requires a user-specified secret key to start, so the deployment must provide one before the first boot.
- SECRET_KEY: Generate a strong value, store it outside the repository and never reuse it across environments.
- PREVIOUS_SECRET_KEY: Set it only while rotating the key, then remove it once the rotation is complete, as the configuration documentation describes.
- Database passwords: Pass the metadata database credentials as secrets, and limit the database account to what Superset needs.
- OAuth client secrets: Store identity provider secrets in the same secrets store, and rotate them on the provider's schedule.
- Access and logging: Restrict who can read the secrets store and make sure startup logs and configuration dumps do not print secret values.
After any change, confirm the secret names and rotation steps with the configuration documentation for your version, and test them in staging first.
In containers and on Kubernetes, the three layers map onto different deployment objects. The committed superset_config.py is typically baked into an image or mounted from a version-controlled ConfigMap, per-environment values come from ConfigMaps or plain environment settings, and secrets come from Secret objects or an external secrets manager. Keeping these objects separate means a config change does not force a secret change, and a secret rotation does not require a new image.
- Image contents: Build the image from committed code only, with no environment-specific or secret values inside it.
- Mounted files: If you mount the config file, pin it to the same revision as the image that runs Superset.
- Secret references: Reference secrets by name from the pod or container definition rather than copying them into manifests.
- Restart behavior: Confirm which changes need a process restart, since environment variables and mounted files are read at startup.
- Helm or operator inputs: If you use a chart or operator, check which values it exposes for secrets and which it leaves to you.
Confirm your chart or operator's secret handling with its own documentation, because the Kubernetes install path has changed across releases.
Configuration patterns and when each fits
This table compares five ways to manage superset_config.py across environments, with the fit, main risk and deployment-specific checks for each. Read it to shortlist one or two patterns for your team, then confirm the details against your deployment method before you commit.
| Pattern | When it fits | Main risk | What to confirm for your deployment |
|---|---|---|---|
| Single file with os.environ lookups | One small instance with few environments and a single owner for config changes | Defaults and non-secret values drift between hosts, and anyone with file access can read values written inline | On a PyPI install on a VM, confirm the file location Superset loads and that the service user can read it but not other users |
| Base file plus environment override modules imported at the end | Several environments (dev, staging, production) that share most settings but differ in a few | Overrides hide behavior, so reviewers must read two files to know the effective value | Confirm the import order your version expects and that the override path is set per environment, not hard-coded in the image |
| Mounted file from a ConfigMap or volume | Kubernetes or Docker deployments where config changes should be reviewed and rolled out without rebuilding the image | Secrets placed in a ConfigMap are readable by anyone who can read the object, and pods may need restarts to pick up changes | On Kubernetes, confirm whether the Helm chart or the Operator supports mounting your file and how a config change triggers a rollout |
| Config baked into a custom image | Teams that want an immutable, scanned artifact per release and few runtime variables | Secrets or environment-specific values end up in image layers and registries, and a rebuild is needed for every change | Confirm that no secret reaches the build stage and that the image is rebuilt per environment or reads values at runtime |
| Secrets manager injecting environment variables | Production environments where secrets must be rotated, audited and kept out of files and images | Secret sync failures can stop startup, and the manager becomes a dependency that must be highly available | Confirm the injection method your platform supports, the rotation process for SECRET_KEY and whether PREVIOUS_SECRET_KEY is set during rotation |
How to set up a versioned superset_config.py
This sequence turns Superset configuration into reviewable code: a secret-free base file in Git, secrets injected at runtime, one environment selector, and a read-only mount that a staging check can prove before production.
Create a secret-free base configuration in Git
Create a
superset_config.pyin a dedicated configuration repository with the settings that are the same in every environment, such as feature flags, authentication method, and role mapping. Keep every password, token, and key out of the file, and do not hard-code the metadata database connection string. Your platform or infrastructure engineers own this file, and a second reviewer should approve changes before merge. The step is done when a secret scanner run against the repository reports no credentials and the file has been merged to the main branch with a review record attached.Load secrets from environment variables and fail fast
Read
SECRET_KEY, the metadata database URI, and other credentials withos.environinside the configuration file, and raise an error at import time ifSECRET_KEYis missing. Superset requires a user-specified secret key to start, so a missing value should stop the process before it serves traffic instead of falling back to a default. Store the values in your secrets manager and inject them into the runtime, not into the image. The step is done when a test start with the variable unset exits with a clear error message that names the missing variable and nothing else.Add an environment override module selected by one variable
Create per-environment override modules, for example one for development, staging, and production, and have the base file import the one named by a single variable such as
SUPERSET_ENV. Keep the differences small: hostnames, log levels, cache endpoints, and the list of enabled features. Make the base file reject unknown values so a typo cannot silently load the wrong settings. The step is done when a unit test loads each environment name and confirms the expected override values, and an unknown name fails.Confirm the metadata database is PostgreSQL or MySQL
Point the production and staging configuration at a PostgreSQL or MySQL metadata database. Superset's architecture documentation says the SQLite default used by Quickstart and PyPI installs is not for production, and the Docker Compose PostgreSQL container is also not recommended for production. Check the supported versions in the configuration documentation before you choose a version, and confirm the chosen release against the Superset version you are deploying. The step is done when the staging configuration resolves to a supported database engine and a connection test succeeds from the application host.
Mount the configuration read-only into the runtime
Mount the configuration directory into the Superset container or host path as read-only, and set the variable that tells Superset where to find the file. Do not bake the file into a layer where it can be changed by a later build without a Git record, and do not mount secrets as files unless your secrets platform requires it. Ask your platform team which mechanism they use to inject environment variables and file mounts so the same pattern applies to every environment. The step is done when a write attempt from inside the running container to the mounted path fails, and the mount shows as read-only in the container's configuration.
Check that the app starts without the key
On staging, start the application with
SECRET_KEYremoved from the environment and confirm that startup stops with the expected error. Then restore the variable and confirm the application starts normally. Repeat the check after any change to the configuration loader. This test proves the fail-fast behavior works in the deployed container and not only on a developer laptop. The step is done when both runs produce the expected results and the outcome is recorded in the change ticket.Scan logs and image history for secrets
Search the container logs from the staging start and inspect the image history with
docker historyor your registry's layer viewer for any credential values. Run a secret scanner against the image filesystem and the configuration repository. Superset should not echoSECRET_KEYor the database password on startup, but confirm this in your own environment because logging depends on your settings and wrappers. The step is done when the search finds no credential values and the findings from the scanner are either empty or have a documented disposition.Confirm the active environment in the running application
Expose the active environment name in a non-sensitive way, such as a startup log line or a read-only status field that your monitoring can read, and compare it with the intended value for that deployment. This catches the most common mistake in multi-environment setups, where a production container starts with the staging selector. Repeat the check after each upgrade of Superset, because the upgrade guide asks you to test in staging first. The step is done when the reported environment matches the target on every deployed instance, and a mismatch triggers an alert.
Handling SECRET_KEY, database credentials and rotation safely
Superset's configuration file is code, so the values that protect sessions and reach the metadata database must live outside it. The cards below cover where secrets come from, how they reach the running process, how they rotate, and how changes are reviewed and kept out of support channels.
Generate and store SECRET_KEY outside Git
Superset will not start without a user-specified SECRET_KEY, a requirement added in version 2.1.0. Keep the value in your secret manager or an orchestrator secret, and have superset_config.py read it from the environment with os.environ. Check that no environment-specific file containing the key is committed, and that the key differs between staging and production so a copied database cannot be used to forge sessions in another environment.
Rotate with PREVIOUS_SECRET_KEY
Superset supports rotating the secret key by setting PREVIOUS_SECRET_KEY alongside the new SECRET_KEY, so that existing sessions and encrypted values can still be read during the transition. Plan the rotation as a change with a window, confirm in staging that logins and database connections still work, then remove the previous key once the transition is complete. Confirm the exact re-encryption steps for your version in the configuration documentation before you begin.
Keep database and OAuth credentials out of the image
Connection strings for the metadata database, Redis and identity providers, plus OAuth client secrets, should be injected at runtime rather than baked into a container image. An image built with credentials can be copied from a registry or scanned output and leak values for every environment that pulls it. Confirm that build arguments and image layers contain no secrets, and that each environment supplies its own values at start.
Restrict who can read the mounted file
Where superset_config.py is mounted into a container or host path, its readable scope is the set of people and service accounts who can read the secret values it loads. Limit read access to the Superset service account and the platform team, set file permissions so other local users cannot read it, and avoid mounting the file into sidecar or debugging containers that do not need it. Review these permissions whenever the deployment method changes.
Audit changes to secret stores
Every change to a secret, whether a new SECRET_KEY, a rotated database password or a new OAuth client secret, should leave a record of who changed it, when and why. Confirm that your secret store keeps version history and access logs, and that a change to a secret is linked to a ticket and a rollout. Auditors often ask for this trail, and it helps you match a failed login or connection error to a specific rotation.
Never print config values in tickets
When a Superset login or database connection fails, support requests often include the configuration file or the startup log. Both can contain secret values or connection strings with embedded passwords. Ask your team to redact values before sharing, to report the variable name rather than its content, and to check that application logs do not echo settings at startup. Treat any value that has been pasted into a ticket as exposed and rotate it.
Managing feature flags and auth mappings as reviewed code
Treat FEATURE_FLAGS and authentication settings such as AUTH_ROLES_MAPPING as access control, not as tuning knobs. A flag can expose a new screen or capability to every user who logs in, and a group-to-role mapping decides which people receive which Superset roles. Both should go through the same pull request, review and approval path as any other change to the application, with the reviewer checking the effect on who can see and do what. A change that alters a role mapping deserves the same scrutiny as a change to a firewall rule, because a small edit can grant broad data access without any visible error.
Keep flag defaults identical across development, staging and production unless a difference is intended and written down. The simplest pattern is one shared base file in Git that sets every flag explicitly, with environment-specific files holding only the documented exceptions. When an environment differs, record the reason beside the setting so an auditor or a new engineer can see that the gap was deliberate. Silent drift is the usual problem: a feature tested in staging appears in production because someone toggled it in one place only. A short diff between the rendered configuration of each environment, run as part of the release check, catches this before users do.
Confirm which flags exist in your exact Superset version and what each one changes before you copy a setting from documentation, a blog post or another team's file. Flags are added, renamed and retired between releases, and the default for a flag can change as well, so a value that was safe on one version may behave differently after an upgrade. Read the flag list and the release notes for your version, record what each enabled flag exposes, and note the flags you have deliberately left off. Add that check to the upgrade review so that a flag change is reviewed as carefully as the code that depends on it. The Superset configuration documentation is the starting point for this review.
Reviewing config changes when you upgrade Superset
What should I diff in superset_config.py before upgrading Superset?
Diff the configuration file in your Git repository between the version you run today and the version you plan to install, then review every key you override and every custom code path your config imports. Focus on settings for the metadata database, SECRET_KEY and PREVIOUS_SECRET_KEY, authentication and AUTH_ROLES_MAPPING, caching and Celery, and feature flags. Confirm that each override still exists in the new release's configuration documentation. Keep secrets out of the diff itself by checking that values come from environment variables, so the review covers structure without exposing credentials.
How does UPDATING.md tell me which config changes an upgrade needs?
The UPDATING.md file in the Superset repository lists breaking changes between versions, and that is the first place to look for configuration keys that were renamed, removed or changed in behavior. Read every entry between your current version and the target version, not only the latest one, because changes can stack across releases. For each entry, check whether your superset_config.py or environment variables reference the affected setting, and record the change in your upgrade ticket. Treat UPDATING.md as the source of truth for breaking changes, and confirm any unclear item against the release notes on GitHub before you deploy.
Why back up the metadata database before I upgrade Superset?
The metadata database holds dashboards, charts, datasets, users, roles and other application state, so a failed migration can leave it in a state you cannot easily reverse. The upgrade documentation tells you to back up the metadata database before running any upgrade step. Take a backup with your database's native tooling, store it outside the upgrade host, and test that you can restore it to a separate instance. Keep the backup until the upgraded system has run successfully for a period your team agrees on in advance.
What do superset db upgrade and superset init do during an upgrade?
The upgrade documentation directs you to run superset db upgrade to apply database schema migrations, then superset init to create default roles and permissions and refresh them for the new version. Run both commands in that order after the backup and before you start the web and worker processes on the new code. Capture the command output in your change record, since errors here usually indicate a schema or permissions problem that should be fixed before traffic reaches the instance.
Why should I test a Superset upgrade in staging first?
Staging lets you find configuration breaks, failed migrations and login or role problems without affecting analysts or scheduled reports. The upgrade documentation recommends testing in staging first, and that test is only useful if staging uses the same superset_config.py structure, the same metadata database engine and similar data connections as production. Use a copy of the production metadata database where your data policy allows it, and check login through your identity provider, dashboard rendering, a sample of scheduled reports and alert delivery. Ask your team to confirm the staging environment matches production before relying on the result.
How do I keep a rollback copy of the previous Superset config?
Store the previous configuration in version control as a tagged commit or release artifact that is tied to the Superset image or package version it worked with. Record the exact image tag or package version, the configuration commit and the metadata database backup in one change record so that you can restore all three together. Do not rely on an editable copy on the server. If you deploy from containers, keep the previous image available in your registry until the upgrade is accepted. Confirm that your rollback steps have been rehearsed in staging.
Can my own team handle a Superset config upgrade, or should a partner do it?
Your team can run a config upgrade if it has someone who reads UPDATING.md carefully, can operate the metadata database and has a staging environment that matches production. A partner is useful when the configuration is large, relies on custom code, or ties into SSO, embedding or caching where a mistake is hard to see. DeployOpen plans and performs upgrades under a written, fixed scope and also offers ongoing upgrade retainers. Ask any provider to show the config diff method, the backup and rollback steps, and the staging test plan in the quote.
Security and operational risks of config drift
Superset configuration is Python code that runs with full application privileges, so a mistake in one environment can expose secrets, weaken login or change behavior without anyone noticing. The cards below cover the most common failure paths and the control that reduces each one.
Secrets committed to Git history
A hardcoded SECRET_KEY or database password in superset_config.py stays in Git history even after it is deleted from the current file. Anyone with read access to the repository, including past contractors and CI systems, can recover it. Reduce this by reading every secret from environment variables or a secrets manager at startup, and by scanning commits for credentials before merge. Check that the config file contains no literal secret values at all.
Overrides that disable auth or security
A staging override that sets a permissive authentication or CSRF setting can be copied into production by a template or a careless merge. Because Superset starts without an error, the change looks harmless. Reduce this by keeping security-relevant keys in a base file that environment overrides cannot replace, and by requiring a review for any change to authentication, session or CSRF settings. Ask which keys are locked in your setup.
Web and worker containers diverge
Superset web servers and Celery workers both load configuration, and they must agree on the caching backend, result backend, SECRET_KEY and feature flags. When one image is rebuilt with a different config file, alerts and reports can fail while the UI still works. Reduce this by building one image, mounting one config source into every role, and comparing the effective settings of each container after deployment.
Unreviewed feature flag changes
The FEATURE_FLAGS dictionary turns on features that may change how users reach data, embed content or export results. A flag switched on for testing can stay on in production with no record of who approved it. Reduce this by keeping flags in version control, requiring a named reviewer for each change, and recording the reason next to each flag. Confirm which flags your release enables by default before you deploy.
Operator alpha API changes
The Apache Superset Kubernetes Operator is in alpha with a v1alpha1 API, and its custom resource schemas may change between releases. If your team mounts superset_config.py through fields defined in those resources, an operator upgrade can move or drop the mount without a visible error. Reduce this by pinning the operator version, reviewing its release notes before each upgrade, and checking the rendered config inside the pod after every change. Confirm the operator's current stability statement with the project before you rely on it.
No config review trail
If configuration changes are made directly on a server or in a container spec, there is no record of what changed, when, or who approved it. Auditors testing change management will ask for that record, and incident responders need it to explain a broken login or a leaked secret. Reduce this by making Git the only path to change configuration, tagging each deployed config version, and keeping the upgrade checks from the upgrade documentation with each change.
Questions teams ask before managing Superset config in-house
How much effort does a versioned Superset config setup take?
The effort depends mostly on how many environments you run, whether a secret store already exists, and how your team deploys Superset. A single production instance with a secret store and an existing pipeline is a small change: move superset_config.py into Git, read secrets from environment variables, and add a review step. Each extra environment adds overrides to test, and a deployment method that is new to your team, such as moving from a VM to containers, adds its own work on how config is mounted. Ask any partner to split the quote into config design, pipeline changes, secret handling and testing, so you can see which part your team already covers.
Can our team manage Superset configuration without a partner?
Yes, if the team has Python and Git skills and a secrets tool that your organization already trusts. The core work is writing a config file that reads its values from the environment, keeping environment-specific settings in separate files or variables, and reviewing every change before it reaches production. The Superset configuration documentation covers the settings, and the upgrade documentation tells you to check UPDATING.md for breaking changes. Teams without anyone who owns Superset internals usually find the upgrade reviews take the most judgment, so plan for a named owner who can read those notes.
What access does the team need to start this work?
You need access to your deployment pipeline, the secret store that holds database credentials and keys, and a staging copy of Superset that mirrors production closely enough to test config changes. Without a staging copy, every config change becomes a production test. You also need read access to the metadata database for backup checks before an upgrade. Confirm who can approve changes to secret values and who can change feature flags, because those permissions decide how much of the review process you can automate.
What recurring work comes with config management?
The recurring work is reviewing superset_config.py and its overrides on every Superset upgrade, checking feature flags before enabling them, and rotating secrets on your own schedule. The upgrade documentation asks you to back up the metadata database, run superset db upgrade and superset init, and test in staging first. Each upgrade may change which configuration keys are needed, so the review belongs in the upgrade plan rather than after it. Budget that as an ongoing task for whoever owns Superset, not a one-time project.
How do we rotate SECRET_KEY without logging users out or breaking sessions?
Superset requires a user-specified SECRET_KEY to start, and the configuration documentation describes rotating it with PREVIOUS_SECRET_KEY. Set the new key as SECRET_KEY and keep the old one in PREVIOUS_SECRET_KEY for the transition, then remove the old value once the rotation is complete. Store both values in your secret store, never in the config file. Test the rotation in staging first, because a wrong value can stop the application from starting or invalidate existing sessions.
Which Superset config changes need the most review?
Changes to secrets, authentication settings, database connection settings, feature flags and anything that affects the metadata database deserve the most review. Authentication changes can lock users out or grant wrong roles, and a missing secret can stop Superset from starting. Feature flags change behavior for every user, so record who approved each flag and why. Settings that depend on the metadata database version should be checked against the supported PostgreSQL and MySQL versions in the configuration documentation before any change goes live.
When is it worth scoping outside help for Superset configuration?
Outside help is worth scoping when you run several environments with separate identity providers, when secrets must stay inside a controlled network with no external lookups, or when your team cannot give the upgrade review enough time. It also makes sense when auditors will test how config changes are approved and logged, because that evidence needs to be built into the pipeline. DeployOpen plans, implements and documents this work inside your infrastructure from a written scope, and ongoing reviews are available as a retainer. Ask for the scope in writing before work starts so the deliverables and timelines are clear.
How DeployOpen scopes Superset configuration work
DeployOpen scopes configuration work for Superset as a fixed, written engagement before any change is made to your environments. The agreed scope names the config layout, the files and their load order, which settings are shared and which are overridden per environment, and where each secret is stored and injected. It also states what the deliverables are, such as a documented superset_config.py structure, an environment variable inventory, mounted config for your container images, and an upgrade review procedure that checks your settings against each release's breaking changes. Because DeployOpen plans, implements, hardens and documents self-hosted platforms inside your own cloud accounts, data centers or restricted networks, the scope also records where the config repository, secret store and metadata database will live and who on your side owns each of them.
Several decisions are best settled during scoping rather than during the build. Confirm who holds the SECRET_KEY and the rotation procedure that uses PREVIOUS_SECRET_KEY, which secret manager your platform team already operates, which feature flags your data governance group has approved, and how a config change gets reviewed and promoted from staging to production. Ask whether your Superset version will be pinned or tracked, and who reads the upgrade notes before each change. These answers shape the written scope, and an accurate scope is what lets the quote reflect the real work instead of an assumption.
Ongoing support is available as a retainer for teams that want DeployOpen to keep configuration current after the initial work. A retainer can cover reviewing configuration changes proposed for upgrades, updating the environment variable inventory as features change, rotating secrets on a schedule you set, and checking that staging and production still match. Ask any provider, including DeployOpen, to define in writing what the retainer includes, how requests are logged, and how quickly changes are reviewed, so that the recurring part of the work is as explicit as the one-time build.
