How Grafana Live streaming panels depend on WebSocket connections
A team running self-hosted Grafana behind a reverse proxy or load balancer often notices that streaming panels stop updating, freeze, or quietly fall back to periodic refresh. The cause is usually not Grafana itself but the path between the browser and the Grafana server. Grafana Live pushes data to open dashboards over a persistent connection instead of waiting for the next query, and that connection is a WebSocket served under the /api/live/ path. If the proxy does not pass the upgrade handshake through, or closes idle connections early, the streaming panel loses its feed while the rest of the dashboard keeps loading normally.
Most teams can leave Grafana Live at its defaults when the Grafana server is reached directly or through a single proxy that already supports WebSockets, and when they only use standard polling panels. Planning becomes necessary when streaming panels are a core part of an operations view, when traffic passes through several layers (a CDN, a web application firewall, an ingress controller and a load balancer), or when Grafana runs as two or more active-active servers. In each case the reader should confirm, in the current Grafana documentation for the installed version, how Live is configured and which proxy settings it requires, because the Grafana docs describe the requirements and their limitations per release.
Before changing anything, check the proxy access and error logs for failed upgrade requests on /api/live/, and check the browser developer tools for WebSocket connections that close after a fixed interval. Record the Grafana version, the proxy product and its version, and the proxy timeout settings for long-lived connections. Then review the Grafana high availability documentation, which notes that Grafana Live has limitations in HA setups, and decide whether streaming panels are a supported fit for your topology before tuning the proxy.
Proxy and deployment options for Grafana Live traffic
This table compares four ways to place Grafana Live traffic in front of Grafana, from a single server to a Kubernetes ingress. Read each row as a question: does your setup need multiple Grafana nodes, and who terminates and forwards WebSocket connections? The main takeaway is that Live streaming works most simply on one node, and multi-node setups need a deliberate plan for Live's documented HA limitations.
| Option | When it fits | What it requires | Trade-off |
|---|---|---|---|
| Single Grafana server behind a reverse proxy | One Grafana node serving a small team or a lab where streaming panels matter | Proxy forwards the /api/live/ path with WebSocket upgrade headers and a read timeout that suits long-lived connections | Simplest to operate and debug, but no node redundancy if the server or proxy fails |
| Proxy with WebSocket pass-through on /api/live/ | Single or multi-node setups where one proxy layer handles TLS for all Grafana traffic | Explicit location rule for /api/live/ with Upgrade and Connection headers, plus idle timeout tuned to your streaming panels | Easy to misconfigure: normal HTTP paths can work while streaming silently fails |
| Load balancer with sticky sessions | Two or more active-active Grafana servers that must share one hostname | Session affinity so a client's Live connection stays on one node, plus shared database for auth tokens | Live limitations in HA still apply, and affinity can unevenly load nodes after a restart |
| Kubernetes ingress | Grafana deployed through the Helm chart or manifests on a cluster you control | Ingress controller configured for WebSocket upgrades and timeouts, with affinity settings matched to the Grafana service | Controller-specific annotations vary, so confirm behavior with your ingress vendor and test a live panel after each change |
Configuring Grafana Live behind a proxy step by step
This sequence aligns the Grafana root URL, every proxy hop and the Live configuration so that WebSocket connections on /api/live/ upgrade, stay open while idle and reach the browser. Each step ends with a check you can run before moving on.
Confirm the root URL and Live settings
The platform owner opens the Grafana configuration file, usually
/etc/grafana/grafana.ini, and checks thatroot_urlunder[server]matches the exact address users type into their browser, including scheme, host, port and any subpath. Then they review the[live]section for allowed origins and connection limits and record the values in the change ticket. Confirm the exact keys and defaults for your Grafana version in the configuration documentation, because they change between releases. Done when the recordedroot_urlmatches the address bar in a browser session opened for the same instance.Map every proxy hop in the path
Trace the request path from the browser to the Grafana process and list each component that handles it: cloud load balancer, Kubernetes ingress, reverse proxy, web application firewall or CDN. Each hop must pass the HTTP Upgrade handshake through to Grafana, and some managed services need an explicit setting to allow WebSockets. Note the product name and version of each hop so you can check its documentation later. Done when you have a written list with one owner per component and no hop left unidentified.
Forward WebSocket upgrades on /api/live/
On an nginx reverse proxy, add a
mapblock that sets$connection_upgradefrom$http_upgrade, then add alocation /api/live/block that usesproxy_http_version 1.1, setsUpgradeandConnectionheaders from those variables, and passesHostthrough. Equivalent settings exist in other proxies, so confirm the syntax in their documentation. Run this command with an authenticated session cookie: `curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" Done when the response starts with HTTP 101 Switching Protocols. A 401 response means the path reaches Grafana but the session is not authenticated.Set timeouts for long-lived connections
A Live connection is open for as long as a browser tab shows a streaming panel, and most of that time it carries little traffic. Proxies with short default read or idle timeouts close these sockets, which looks like a panel that stops updating after a quiet period. For the
/api/live/location, raise the proxy read and send timeouts to values that match how long your users keep dashboards open, and limit concurrent connections per client or per upstream so a single user cannot exhaust the pool. Ask the vendor or load balancer team for the documented defaults and maximums. Done when the settings are in the file and recorded with the reason for each value.Validate the configuration and reload the proxy
Run the proxy's syntax test before applying anything, for example
nginx -ton nginx, and fix every reported error. Reload the proxy rather than stopping it so that existing connections are handled during the change. If you changedroot_urlor any[live]key ingrafana.ini, restart the Grafana service withsudo systemctl restart grafana-serverin a maintenance window that suits your users. Done when the syntax test passes, the proxy reload completes without errors andsystemctl status grafana-serverreports the service as active.Verify a streaming panel in browser developer tools
Open a dashboard that uses a Live channel, such as a panel backed by a streaming data source or plugin you have installed, and confirm which channels your version provides. In the browser's developer tools, open the Network tab, filter for WS and select the connection to
/api/live/ws. Its status should show 101 and its Messages or Frames view should show data arriving as the panel updates. Done when the panel refreshes without a page reload and the socket stays open through the period you configured.Repeat the checks on every access path
Users may reach Grafana through a VPN, a corporate network, a partner network or an internal ingress with a different hostname, and each path can pass through different proxies. Repeat the curl handshake and the browser check from each path and record the result. Pay particular attention to hostnames that differ from
root_url, because Live can behave differently when the origin does not match the configured value. Done when every documented access path returns 101 and shows live updates, and any path that fails has a named owner and a fix.
What Grafana Live does not do well in highly available setups
Grafana Live pushes updates over a persistent connection to one node, so high availability changes what a live panel can expect when a node fails or a user's session moves. The cards below cover the documented HA limitations for two or more active-active Grafana servers behind a load balancer and the database that backs them.
Live streams are tied to one node
A WebSocket connection from a browser terminates on one Grafana server, and that server holds the live channel state for the panels it serves. Requests that load balance to a different node do not automatically share that stream. Check how your load balancer distributes WebSocket traffic and whether a live panel reconnects cleanly when its node is removed from the pool. Treat the documented HA limitations for your version as the starting point for this design.
Node failure drops live updates
When one active-active node fails, browsers connected to it lose their live channel and must reconnect through the load balancer to a surviving node. Panels may show stale values or a gap until the reconnect completes. Ask your team how long a reconnect takes in your environment, how the dashboard signals a dropped stream to the viewer, and whether operators are expected to refresh manually. Test this with a planned node drain before relying on it for operational displays.
Sessions moving between nodes
Database-backed auth tokens let users reach any node, but that does not mean a live channel moves with the session. A user who is routed to a different node after a load balancer change or a timeout starts a new live subscription. Confirm that your load balancer uses a policy that keeps a browser on one node where possible, and document what happens to panels when stickiness is lost. Session behavior should be verified separately from live streaming.
Alerting and live views are separate concerns
The HA documentation notes that alerting needs additional configuration to avoid duplicate notifications. Live panels do not replace alert delivery, so an operator who depends on a live view for a critical signal should also have an alert route that does not rely on a browser connection. Confirm the notification design for each alert rule and avoid assuming that a live panel is a monitoring control.
Proxy and connection limits affect stability
Reverse proxies, ingress controllers and load balancers each impose idle timeouts, connection caps and buffering on long-lived connections. A timeout shorter than the expected idle period causes repeated reconnects that look like node failures. Record the timeout values at every hop, compare them with Grafana's live connection behavior in a test, and check whether your team owns each hop or whether a cloud or network group does.
Verify behavior in your own environment
Read the current HA limitations section for the Grafana version you run, because limitations change between releases and this page does not replace that text. Then test node drain, node failure, load balancer reconfiguration and a database failover while a live dashboard is open. Record what the viewer sees and how long recovery takes, and decide which live panels are acceptable for your operations and which need a different data path.
Questions teams ask about Grafana Live and WebSockets
Do Grafana Live panels work behind a standard reverse proxy?
Often yes, but only when the proxy forwards the WebSocket upgrade for the live endpoint. Grafana Live streams data over /api/live/, and a proxy that treats that path as ordinary HTTP will drop the connection or fall back in ways that look like a broken panel. Check that the proxy passes the Upgrade and Connection headers for that path, that it keeps the connection open longer than its default idle timeout, and that it does not buffer the stream. Confirm the exact header and timeout settings against the Grafana documentation for your version and your proxy's own WebSocket guide before you change anything in production.
What does a proxy configuration change cost in effort, and who makes it?
The change itself is usually small: a few directives on the location block for the live path, plus a reload. The cost comes from access and verification. Someone must have rights to edit the proxy, which may sit with a network or platform team, and that team must review the change against its own standards. Your internal team can make the edit if it has that access and a staging node to test against. A partner is useful when the proxy is shared with other applications, when several proxies sit in series, or when the change has to be proven and documented for an audit file.
How long does verification take, and what makes it longer?
Verification length depends mostly on how many layers sit between users and Grafana. A single proxy in front of one instance is a short test: open a live panel, watch it update, then confirm the connection survives an idle period longer than your proxy timeout. Add a load balancer, a second proxy, a CDN, or a Kubernetes ingress, and each layer needs its own check. Multiple Grafana nodes add sticky-session and HA tests. Ask any provider to break the verification into the layers in your path so you can see where the time goes.
What must our team provide before an outside engagement starts?
Plan to provide the following before work begins:
- Administrative access to the proxy or ingress configuration for a staging environment, and read access to production configuration
- Certificates and the hostnames that users and the proxy use, including the chain your organization trusts
- Test user accounts with Grafana roles that can open dashboards containing live panels
- A staging Grafana node that matches production's version, database and network path
- Your change window rules and the name of the person who approves production changes
Without a staging node, verification usually moves into production, which raises the risk of the change.
Can our own team configure and test live streaming without help?
Yes, if your team has someone who can edit the proxy, read Grafana's logs, and reproduce a live panel from a browser's developer tools. The work is a sequence of checks: confirm the upgrade request reaches Grafana, confirm the response is a switch to the WebSocket protocol, then test long idle sessions and failover. Teams with a single proxy and one Grafana node usually manage this in-house. Teams with several proxies, an HA pair and strict change control often ask for a written test plan first, then run it themselves.
What drives the cost of an outside engagement for this work?
The main drivers are the number of hops between users and Grafana, whether the HA design and its documented limits must be assessed, how many environments need changes, and how much evidence the work must produce. Quotes should show who is paid for planning, configuration, testing, and handover, and whether each is a one-time fee or part of a recurring support arrangement. DeployOpen works from a written scope and does not publish a price list, so ask for a quote that lists each layer, each environment, and each deliverable, including the test record and the runbook your team will keep.
What limits does Grafana Live have in a high availability setup?
Grafana's high availability documentation states that Grafana Live has limitations in HA deployments, so a multi-node setup needs a planned check before you rely on live panels. Confirm which limitations apply to your Grafana version, how your load balancer routes WebSocket connections, and what users see when a node restarts or a connection moves. Treat live streaming as a feature to validate in your own environment, not one to assume works across nodes.
When to plan Grafana Live work as part of a wider deployment
Plan Grafana Live streaming as part of the broader Grafana deployment or high availability rollout, not as a late add-on. Live streaming usually surfaces when a team starts running Grafana behind a load balancer or reverse proxy, and at that point the proxy, database and session design have to be decided together. A WebSocket path that works on one node can fail once users land on different nodes, and the fix often depends on choices already made for the database and the load balancer.
The high availability model sets the baseline. Grafana's high availability setup uses two or more active-active servers behind a load balancer, sharing a MySQL or PostgreSQL database, with database-backed auth tokens so a user can reach any node. Confirm that your database tier is the production-grade option the installation docs require, because SQLite is not supported for this design. Then check how your load balancer handles sticky sessions and long-lived connections for /api/live/, since that decision affects how users experience live panels after a node restart or failover.
Alerting needs its own planning in the same work. The high availability documentation notes that alerting requires additional configuration to avoid duplicate notifications, and that Grafana Live has limitations in HA. Read the current Grafana high availability guidance for the exact limitations in the version you plan to run, and test live panels against your real proxy chain before you commit the design to production.
Verifying live panels after a proxy or HA change
This sequence gives you a repeatable check that streaming panels still connect and recover after a proxy or high availability change, with results recorded for change control. It shows whether each path works before users find out.
Confirm the baseline and the change scope
Before testing, the change owner records which proxy rules, load balancer settings and Grafana nodes changed, and lists every route that serves the Grafana Live endpoint under /api/live/. Identify one dashboard with a streaming panel that a tester can open and watch for live updates. The step is done when the change record names the nodes, the proxy layers and the test dashboard, and the team agrees on what "live" looks like on screen.
Open the streaming panel through each node
The tester opens the test dashboard by connecting directly to each Grafana node, bypassing the load balancer, and watches whether the streaming panel receives updates. Repeat this for every node in the pool, using the same browser and user account. The step is done when each node's result is written down as updating, stalled or failed, with the time and node name.
Open the streaming panel through the load balancer
The tester opens the same dashboard through the public or internal address that users actually use, and confirms that the panel updates. Repeat the check several times so that requests land on different nodes, since a session that sticks to one node can hide a fault on another. The step is done when every attempt is recorded and any attempt that stalls is noted with the node it reached, if the proxy shows that.
Simulate a node restart during a live session
With the panel open and updating, an operator restarts one Grafana node in a non-production window, or in a staging copy of the topology. The tester watches the panel and records whether it reconnects on its own, shows a visible disconnection, or stops updating without warning. Repeat with each node that serves traffic. The step is done when the reconnection behavior for each restart is recorded alongside the time the panel resumed updates.
Check proxy logs for upgrade failures
The operator reviews the access and error logs of each proxy or load balancer layer for the test window and looks for WebSocket upgrade requests on /api/live/ that returned an error code, were closed early, or were not upgraded. Compare log entries with the tester's timestamps. The step is done when each upgrade attempt has a matching log line or the team notes that logging does not record it, which is itself a finding to resolve.
Record results and set pass criteria
Your team defines pass criteria before the test, such as which behaviors are acceptable after a restart and how many failed attempts are tolerable, since these depend on your users and service expectations. The change owner records each node, path, restart and log result against those criteria, names the tester and the date, and attaches the log excerpts. The step is done when the record shows pass, fail or accepted-with-risk for each check and is filed with the change ticket.
Re-test after any fix and close the change
If any check failed, the team corrects the proxy or Grafana configuration, then repeats the affected checks rather than only the one that failed, because a fix on one path can affect another. Confirm the limitations of Grafana Live in your HA design are still documented for the users who depend on the panels. The step is done when the final record shows all required checks passing or accepted, and the change is closed.
Official Grafana sources for HA and proxy configuration
Which official Grafana page covers high availability setup for Grafana Live?
The Grafana high availability setup page is the primary source for this topic. It describes running two or more active-active Grafana servers behind a load balancer, sharing a MySQL or PostgreSQL database, and using database-backed auth tokens so users can reach any node. It also notes that Grafana Live has limitations in HA setups and that alerting needs extra configuration to avoid duplicate notifications. Read it before you choose a load balancer design, because the Live limitations determine which panels and streaming behavior you can promise to users. Confirm the current wording against the version you plan to run, since the limitations may change between releases.
Where do I find the supported installation packages and database requirements for Grafana?
The Grafana installation documentation lists packages for Debian and Ubuntu, RHEL and Fedora, SUSE, macOS and Windows, along with Docker images and Helm charts for Kubernetes. It also states the database options: SQLite 3 by default, MySQL 8.0 or later, or PostgreSQL 12 or later. The same page says SQLite does not scale for production, so an HA design needs MySQL or PostgreSQL. Use it to confirm the reverse proxy and server settings that your install method supports, including how the root URL and protocol are configured when Grafana sits behind a proxy. Check the page for the version your environment targets before you write the proxy configuration.
Does Grafana Live work the same way in a single instance as in a high availability cluster?
No, the documented HA limitations mean Grafana Live should be tested separately in a clustered setup. A single Grafana server handles live streams on one node, while a load-balanced cluster spreads client connections across several servers. The HA page is the authoritative place to confirm which Live features are affected in your version. Plan a test that opens streaming panels through the load balancer, then restarts or drains one node and observes what users see. Record the results so the team knows which dashboards should use polling instead of live streaming.
Which parts of the Grafana installation guide matter most for a reverse proxy in front of Grafana?
The installation guide and the HA page together cover the settings that a proxy in front of Grafana depends on. Focus on the server settings for the public root URL, the protocol and the serve-from-sub-path behavior, since WebSocket paths under /api/live/ are built from these values. Confirm which HTTP headers your proxy must forward and whether the installation method you chose changes the default configuration file location. Document the final grafana.ini or environment variable values in your own configuration repository so the proxy and Grafana settings stay aligned after upgrades.
Who should own the official Grafana documentation review for this project?
The platform or infrastructure team that operates the reverse proxy and the Grafana service should own the review, with a named engineer responsible for checking each documented setting against the running version. DeployOpen can plan and implement the configuration as part of a fixed-scope engagement, with the written scope stating which documentation checks, proxy changes and test runs are included. Your team keeps ownership of the load balancer, network policy and change approval. Ask any partner to show which documentation version they used and how they will record deviations from the official guidance.
How often should we re-check the official Grafana guidance after the initial setup?
Re-check the guidance at each upgrade and whenever you change the proxy, database or load balancer. Grafana's upgrade guidance says major releases come once a year in April or May, minor releases every other month and patch releases at least monthly. Each minor release is supported for 9 months and the last minor release of a major version for 15 months. Those support windows set the cadence for your review, so schedule a documentation check against the current minor release before each planned upgrade window.
Can I rely on the Grafana docs alone to build a production HA setup for streaming panels?
The docs give the authoritative baseline, but they do not replace testing in your own network. They describe the architecture, supported databases and known Live limitations, while your proxy timeouts, connection limits, certificates and client mix determine real behavior. Use the HA page and installation documentation as the reference for each design decision, then verify the result with load and failover tests. Ask any implementation partner to tie each configuration choice to a documented source and to a test result you can review.
How DeployOpen scopes and delivers Grafana Live and HA work
DeployOpen treats Grafana Live as a proxy and topology question before it is a dashboard question. We scope the WebSocket path from browser to Grafana, build and test it in your own infrastructure, and document the limitations that remain in your high availability design.
How we scope
- The written scope lists every hop that carries /api/live/ traffic, from browser to load balancer to each Grafana node.
- We record the proxy products, timeouts and connection limits your team already runs, and which of them the build will change.
- The scope states which Grafana Live features your panels depend on and which high availability limitations apply to them.
- Engagements are fixed-scope, so the scope, deliverables and timelines are agreed in writing before work starts.
How we build
- We configure WebSocket upgrade headers for /api/live/ on your reverse proxy or load balancer and record each change in version control.
- We set proxy connection and idle timeouts, then test long-lived streaming panels against the limits you have agreed.
- We test a multi-node setup, confirming that live panels reconnect cleanly when a user lands on a different node or a node restarts.
- We check the browser console and Grafana logs for failed upgrades, so that fallback behavior is confirmed rather than assumed.
How we hand over
- You receive the proxy configuration, Grafana settings and test results, with each change tied to the topology it was tested on.
- The documentation states the Grafana Live limitations in your high availability design and the affected dashboards for each.
- We write a runbook covering symptoms of a broken upgrade path, the checks to run and who in your team owns each step.
- Your team gets a walkthrough of the tests, so they can repeat them after upgrades or proxy changes.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer, so proxy and Grafana changes are retested together.
- When Grafana releases a new version, we review its guidance for Live and the proxy path before you upgrade.
- Retainer work can cover re-running the WebSocket tests after changes to load balancers, certificates or network paths.
- Any change to the Live design, such as moving to a different high availability topology, is scoped in writing before it is built.
