Grafana configuration with environment variables and secrets

Grafana configuration with environment variables and secrets: when teams need it

A GF_ environment variable overrides one grafana.ini setting at startup, and a __FILE variant reads that value from a file so the password itself never appears in the environment. Teams need this when one Grafana build moves through development, test and production, and the settings that differ per environment, such as the public URL, the database host and the administrator password, must change without editing the image or the base configuration. Grafana reads grafana.ini from its configuration directory, then applies environment variables on top at startup, so the file holds the shared defaults and the environment holds what changes between environments. That layering is what keeps per-environment configuration manageable, because the same artifact can be promoted unchanged.

The layering has limits that matter for planning. An environment variable can only override a setting that already exists in grafana.ini, so it cannot introduce a new option, and a typo in a variable name is silently ignored. Teams should confirm each override against the configuration reference for their exact Grafana version, and check the effective value after deployment rather than assuming the variable took effect. Database credentials and the administrator password are the first candidates for secret files, because they would otherwise appear in container inspection output, process listings and deployment manifests.

Plan the split before the first deployment: decide which values are environment-specific, which are secrets, and which belong in version control as plain files. Keep the non-secret base configuration and the provisioning files in Git, review every change through the same process used for application code, and inject secrets from the platform's secret store at runtime. The Grafana provisioning documentation covers the data sources and dashboards that should travel with this configuration. Confirm with your security team where secret values are stored, who can read them, and how rotation is performed, because the configuration pattern is only as strong as the store behind it.

Which configuration method fits which setting

This table compares the five ways to configure self-hosted Grafana: the grafana.ini file, GF_ environment variables, __FILE secret variables, provisioning files, and as-code tools. Read each row as a question of where a setting belongs and who should be allowed to change it. The main takeaway is that environment variables override existing settings but cannot add new ones, so provisioning and secret files cover the rest.

Method	What it configures	Where it lives and who changes it	When it fits
grafana.ini file	Server, database, security, authentication and other core settings documented in the installation configuration	On the host or in the image at the default config path; changed by the team that owns the server build	Baseline settings shared by every environment, reviewed in version control
GF_ environment variables	Overrides for existing grafana.ini keys, named with the GF_ prefix and the section and key	Set in the container, service unit or Kubernetes manifest; changed during deployment by the platform team	Per-environment values such as root URL or database host, without editing the file
__FILE secret variables	Sensitive values such as passwords, read from a file path supplied through a variable ending in __FILE	Secret mounted into the container or host; changed by whoever rotates the secret store	Passwords and tokens that must stay out of environment listings and version control
Provisioning YAML or JSON	Data sources, dashboards, plugins and alerting resources, loaded from files at startup	Files in the provisioning directory; changed through pull requests by the team that owns dashboards and alerts	Objects that teams should create the same way in every environment and review as code
Terraform or Git Sync	Resources and workflows layered on top of a running instance, as documented by Grafana for as-code management	Separate repositories and pipelines; changed by platform or automation owners	Larger estate management where a second tool already governs infrastructure. Confirm supported resources and versions before adopting

Separating dev, test and production configuration

Most of a Grafana configuration should be identical in development, test and production. Keep these settings in one base grafana.ini (or the equivalent file your packaging uses) that every environment starts from, so that a behavior difference between environments means a deliberate override rather than drift. Settings that define how Grafana behaves as a product, such as the security hardening choices, the logging format, plugin policy and default paths, belong in the baseline. Treat the baseline as the reference that the other environments are compared against during review.

Security posture: Cookie, Content Security Policy and version-exposure settings should match across environments, so that a test result reflects what production does.
Logging and paths: Log format, log destinations and data directories should be the same, which keeps troubleshooting steps portable between environments.
Plugin policy: Allowed plugin lists and install behavior should not vary by environment without a recorded reason.
Feature choices: Enabled or disabled features should be set once in the baseline and changed only through a reviewed change.
Version pinning: The image or package version should be the same artifact promoted through each environment.

Before promoting a change, confirm that the baseline diff between environments is empty except for the values you intentionally set.

Some values must differ by environment, and Grafana's GF_ environment variables are the usual way to set them without editing the base file. The variable name follows the section and key in grafana.ini, so a setting in the server section becomes a variable with a GF_SERVER_ prefix. Typical per-environment values are the public root URL, the database host, and the authentication settings that point at each environment's identity provider. The values below are illustrative only and will differ in your environment.

Root URL: GF_SERVER_ROOT_URL is set to a dev, test or production hostname so that links and OAuth redirects resolve correctly.
Database host: GF_DATABASE_HOST and related database keys point each environment at its own database instance.
Authentication: Identity provider settings, such as the client ID or the allowed domain, are set per environment, while the protocol choice stays in the baseline.
Environment label: A variable can mark which environment is running, which helps operators confirm they are on the right instance.
Limits on scope: GF_ variables override existing settings only. They cannot introduce a key that Grafana does not already define, so every override must match a real setting in your Grafana version.

Confirm each override against the documentation for the version you run before adding it to the environment definition.

Passwords, client secrets and API tokens should never sit in the base file or in the same version-controlled overrides as ordinary settings. Inject them at runtime from a secret store, and point Grafana at them through the __FILE suffix convention for environment variables, which reads the value from a mounted file. This keeps the secret out of the environment listing, the container definition and the repository. Use a separate secret per environment so that a test credential cannot authenticate against production.

Separate secrets per environment: Each environment gets its own administrator password, database credentials and identity provider client secret.
File-based injection: Mount secrets as files and reference them with __FILE variables, so the value is read at startup.
Source of truth: Keep secret values in your organization's secret manager or in the orchestrator's secret object, and record only the secret names in the repository.
Rotation: Plan rotation per environment, so a rotation in test does not require a production change, and confirm that Grafana reads the new value after a restart.
Access control: Limit who can read each environment's secrets, separately from who can deploy configuration changes.

Check that no secret value appears in container inspection output, deployment manifests or pipeline logs.

Teams usually choose between two patterns. The first is a single complete configuration file per environment, which is easy to read in full but duplicates shared settings and invites drift. The second is a shared base plus many small overrides, usually GF_ variables or a short overlay file, which keeps the common settings in one place but spreads the effective configuration across several sources. Neither pattern is correct for every team, and the right balance depends on how many environments you run and who reviews changes.

Full file per environment: Reviewers see the whole effective configuration in one place, but a fix applied to one copy can be missed in the others.
Base plus overrides: Shared behavior changes once, but operators must combine several sources to know what is running.
Many small variables: Each override is easy to audit, but a long list in a deployment manifest can obscure which values are intentional.
Review burden: Decide who approves overrides and whether a change to the baseline needs a wider review than an environment change.
Tooling: Templating or overlay tools can generate the effective file for review, if your pipeline supports it.

Whichever pattern you pick, generate or record the effective configuration for each environment so an auditor can see exactly what ran.

Setting up a version-controlled Grafana configuration

This sequence turns an ad hoc grafana.ini into a layered configuration in which the file holds shared defaults, environment variables hold per-environment values, secrets live outside the repository, and provisioning files are reviewed in Git. The result is an instance whose effective settings you can explain and reproduce.

Inventory the current grafana.ini settings
Start by listing every setting in the live grafana.ini and every environment variable already set on the Grafana process or container. Your platform team should also record which settings were changed from the packaged defaults and who changed them. Group the list into server, database, authentication, security, plugin and logging settings so each group has an owner. You are done when every existing setting is accounted for in a single inventory and nothing is left as an undocumented manual edit on the host.
Separate defaults from environment values
Sort the inventory into two sets. Shared defaults, such as logging format, feature toggles and plugin policy, stay in the grafana.ini file kept under version control. Values that differ between development, test and production, such as the root URL, database host and authentication endpoints, move to environment-specific sources. Note that Grafana environment variables with the GF_ prefix override a matching setting in grafana.ini, but they cannot create settings that the file format does not define. Confirm this against the installation and configuration docs for your Grafana version before you design the split.
Move passwords and database credentials into secret files
Remove every password, client secret and database credential from grafana.ini and from plain environment definitions. For Docker, Grafana supports the __FILE suffix on GF_ variables, which reads the value from a mounted file, so you can point a variable such as the admin password at a file the platform injects. On Kubernetes, store the values in Secrets and mount them, and confirm that your cluster encrypts Secrets at rest because they are stored unencrypted in etcd by default. Your security team should approve the secret store and the rotation owner. You are done when a search of the repository and the deployment manifests finds no credential values.
Set GF_ overrides for each environment
Write one environment file or manifest overlay per environment that sets only the GF_ overrides for that environment, such as the server root URL, database connection and authentication provider. Keep the names identical to the grafana.ini section and key they override, using the uppercase GF_section_key pattern from the configuration docs, and check each name against the docs for your version. Ask the person who reviews the overlays to confirm that no overlay introduces a setting name Grafana does not recognize, because an unknown name is silently ignored. You are done when each environment has a short, reviewed overlay and the shared file is identical across them.
Store provisioning files in Git
Move data sources, dashboards, plugin declarations and alerting resources into provisioning files in the same repository, using the YAML or JSON formats Grafana loads at startup. Where your team wants a stronger as-code workflow, review the other options Grafana documents, such as Terraform and Git Sync, and decide before you build a second system. Require pull request review for every change and tag each release of the configuration. You are done when a clean checkout of the repository plus the environment overlays and secret references is enough to describe the whole instance.
Deploy and restart the instance
Apply the configuration in a non-production environment first, then restart Grafana so the process reads the new file, overlays and secrets. Settings are read at startup, so a change that is not followed by a restart does not take effect. Record the Grafana version that runs in each environment and check it against the Grafana releases page and the upgrade guidance, since patch releases come at least monthly and the configuration syntax can change between versions. You are done when the restart completes without errors in the logs and the running version matches the one you tested.
Verify the effective configuration
Confirm that the running instance reflects the intended values. Review the effective settings through the administration interface or the server settings endpoint, which your team should restrict to administrators, and compare them with the repository. Then change one non-secret value in a test environment, restart, and confirm the new value appears, which proves the override chain works end to end. Check that secret values are not echoed in logs or the interface. You are done when the comparison shows no differences and the test change behaves as expected, with the results saved alongside the release tag.

Protecting secrets and config files in Git

Grafana configuration lives in files, environment variables and Kubernetes objects, and each of those can leak credentials or quietly weaken a security setting. The cards below cover where secrets should and should not sit, who can read them, how they get rotated, how security-relevant changes are reviewed and how the running configuration is checked against what the repository says.

Keep passwords out of Git

Commit the configuration files that set Grafana behavior, but never the values of the admin password, database password, OAuth client secret or API keys. Reference secrets by name, using the __FILE suffix on GF_ variables or a secret reference in the deployment manifest, so the repository holds the pointer and not the value. Add a secret scanning check to the pipeline so a plain-text password fails the commit before it reaches the shared branch.

Keep plain-text values out of pod specs

A value written directly into a container environment block in a Deployment is visible to anyone who can read that object and often ends up in exported manifests and dashboards of the cluster tooling. Mount credentials from a Kubernetes Secret instead, and check rendered manifests from your Helm values for literal passwords. Confirm which teams can run kubectl get against the namespace, because that read access is the same as reading the values.

Restrict who can read Secrets

The Kubernetes good practices documentation notes that Secrets are stored unencrypted in etcd by default. Enable encryption at rest for the API server, then limit get, list and watch on Secret objects in the Grafana namespace to the service accounts and operators that need them. Review role bindings after every team change, because a broad read grant on the namespace exposes Grafana credentials to everyone bound to it.

Rotate credentials without downtime

Plan rotation for the admin password, database user, OAuth client secret and any data source tokens as separate events with an owner and a date. Store the new value in the secret store first, then roll the Grafana pods so they read it at startup, and revoke the old value only after the new one works. Test the sequence in a non-production instance, since a rotated database password that the running pods still hold will stop logins and alerting at the same time.

Review security setting changes

Treat changes to cookie settings, the Content Security Policy, Trusted Types, version exposure and metrics endpoint authentication as security changes that need a second reviewer. A small diff such as a switched-off secure cookie flag or a loosened policy can pass a functional test while weakening the instance. Require pull request review for any file or variable that controls these settings, and record the reason in the commit message.

Audit the effective configuration

The file in Git is not always the configuration a running instance uses, because environment variables, mounted files and image defaults can override it. Periodically read the settings the running process reports, compare them with the repository, and alert on drift. Include the security settings and the list of provisioned data sources, since a value applied from an unreviewed source is the most common reason a deployed instance differs from its reviewed configuration.

Questions teams ask before configuring Grafana for production

Can our own team configure Grafana settings and secrets without a partner?

Yes, most platform teams can do this in-house if they already run containers or Kubernetes and have a secret store. The work is mostly well-documented: setting GF_ environment variables, using the __FILE suffix for passwords, and writing provisioning files. The harder part is agreeing on the rules: which settings live in version control, who approves changes, and how each environment differs. Teams without a clear owner for those decisions, or without someone who has handled secret rotation before, often find the design takes longer than the typing. A partner can help with the design and handover, or your own engineers can build it after reading the Grafana configuration and provisioning documentation.

What makes the effort larger or smaller for a Grafana configuration project?

Four factors drive effort the most:

Number of environments: each extra environment (development, staging, production, a restricted network) adds a set of overrides and a promotion path.
Secret stores: a Docker secret, a Kubernetes Secret, or an external store such as a vault changes how credentials reach the container and who can read them.
Runtime: Docker, Kubernetes with the Helm chart, or virtual machines each handle grafana.ini and environment variables differently.
GitOps tooling: storing configuration in Git, with review and automated deployment, adds pipeline work but reduces drift.

An environment that needs only one secret source and one Git repository is a much smaller job than a multi-cluster setup with several secret stores.

How long does Grafana configuration and secrets work usually take?

Duration depends on scope, and no fixed figure applies to every environment. A single runtime with one secret source and an existing repository is a short piece of work. Adding several environments, an external secret store, provisioning as code for data sources and dashboards, and a review process for changes extends the schedule. Approval cycles on your side, such as change boards and access requests for the secret store, often set the pace more than the technical work does. Ask any provider for a timeline that lists the environments, secret sources and approval steps it assumes, and check each one against your own calendar.

What do we need to provide before the work starts?

Plan to provide these items before work begins:

Access to the secret store where production credentials live, with a named contact who can grant read rights to the Grafana workload identity.
A Git repository for configuration, with branch protection and a defined review path.
A list of environments, runtimes and the database Grafana will use.
Named approvers for production changes and a window for testing.
Access to a non-production instance where changes can be verified before they reach users.

Missing any of these usually shows up as idle time later in the project.

What recurs after Grafana goes live with this configuration?

Several tasks recur after go-live. Grafana publishes patch releases at least monthly and minor releases every other month, and each minor release is supported for 9 months, according to the Grafana upgrade guidance. For a container deployment, upgrading usually means changing the image tag and re-testing configuration. For a Helm deployment, it means changing the chart version and values. Secret rotation, reviews of grafana.ini changes, and checks that environment overrides still match the documented settings also repeat. Confirm with your provider who performs these tasks and how they are scheduled.

What does a fixed-scope quote for this work cover?

A fixed-scope quote, which most DeployOpen engagements use, states the scope, deliverables and timeline in writing before work starts. For this topic, expect the written scope to list the runtimes covered, the environments, the secret sources and how secrets reach the container or cluster, the repository structure and review process, the verification steps, and the handover documentation. Ask whether it includes the upgrade procedure, how changes to scope are priced, and what follow-on support is available as a separate retainer. Pricing is calculated from that written scope, and DeployOpen does not publish a price list.

Where does Grafana store secrets safely, and is Kubernetes Secret storage enough?

Kubernetes Secrets are a starting point, but they need extra controls. According to the Kubernetes secrets good practices page, Secrets are stored unencrypted in etcd by default, so administrators should enable encryption at rest, restrict list and watch access with least-privilege RBAC, or use an external store through the Secrets Store CSI Driver. Docker deployments can use the __FILE suffix to read values from mounted files instead of environment variables. Confirm with your security team which store is approved for production credentials before you choose the pattern.

How DeployOpen scopes configuration work for your platform

A scoping conversation for Grafana configuration starts from what exists today. We look at the grafana.ini file, any GF_ variables set on containers or VMs, and the places secrets currently live, such as shell profiles, CI variables, Kubernetes Secrets or a password manager. We also ask how many environments you run (development, test, staging, production), which of them are separate networks, and who can change each one. The output is an inventory of every setting that differs between environments, along with the settings that should never differ. That inventory decides how much of the work is consolidating existing values and how much is building a new structure from scratch. You should be ready to share the Grafana version, the database in use and the deployment method so the scope reflects what your team actually runs.

The second part of scoping is the repository model. We confirm where configuration will be versioned, how environment-specific values are separated from shared defaults, and how changes are reviewed and promoted. Provisioning files for data sources, dashboards and alerting can live in the same repository as the configuration, which lets a reviewer see a dashboard or contact point change in the same pull request as the setting that enables it. Secret values stay out of the repository. The scope should name the secret source your organization uses, whether that is a Kubernetes Secret delivered by your platform team, a mounted file consumed through the __FILE suffix, or an external store your security team already operates, and it should state who holds the keys and rotation responsibilities.

Engagements of this kind are fixed-scope. Before work starts, we agree in writing on the environments covered, the repositories and pipelines involved, the secret sources to integrate, the verification steps and the handover material. Each deliverable is listed with its acceptance check, so your team knows what has been delivered when the work closes. Costs follow from that written scope, and we do not publish a price list, so ask for a quote built from the inventory and the deliverables. Because Grafana, its plugins and its platform dependencies change over time, many teams also want ongoing support. That is available as a retainer covering reviews of configuration changes, upgrade preparation and periodic checks. Confirm with us at scoping which of these you need, since a retainer and a one-time engagement are priced and written differently.

How DeployOpen delivers Grafana configuration and secrets

DeployOpen works on Grafana configuration as a set of files your team can review, version and redeploy. We separate what belongs in grafana.ini, what belongs in GF_ environment variables and what belongs in a secrets store, then deliver that split for each environment you run.

How we scope

Inventory every grafana.ini setting your team relies on and map each one to a file, an environment variable or a secret
Agree which environments (development, staging, production) need their own values and who owns each change
Confirm the Grafana version and edition, since some authentication options differ between open source and Enterprise
Set the scope in writing before work starts; engagements are fixed-scope and ongoing support is available as a retainer

How we build

Write environment variable mappings with the GF_ section and key naming, and test that each override takes effect at startup
Wire database and admin passwords through the __FILE suffix or Kubernetes Secrets, with no credentials in images or repositories
Define provisioning files for data sources and dashboards so configuration is repeatable across environments
Check that settings Grafana does not read from environment variables are kept in the ini file instead

How we hand over

Store configuration, environment files and manifests in your version control system with review and branch rules you choose
Document which setting lives where, the precedence Grafana applies and the steps to rotate a secret
Run a restart and rollback exercise in each environment and record the results for your operations team
Provide a runbook for adding a setting, changing a secret and confirming the running configuration matches the repository

How we support

Review configuration changes when Grafana releases a new minor or major version, using the upgrade notes your team receives
Re-test secret rotation and provisioning after platform changes such as a new secrets store or cluster upgrade
Offer ongoing support, upgrades and continuous testing as a retainer, priced from the written scope
Confirm with your security team which hardening settings, such as cookie and Content Security Policy options, must stay in the managed configuration

Grafana configuration with environment variables and secrets