Embedded analytics for energy and utilities

Embedded analytics for energy and utilities: where the grid boundary decides the design

The design of a utility dashboard portal is set by which systems fall under NERC CIP and NIS2 scope, not by the charting tool. A customer who wants interval usage and an operator who wants outage and load views may log in to the same portal, but the data behind those views sits on opposite sides of the grid boundary. The usual open-source starting point is Apache Superset with ClickHouse as the analytics store, with Metabase as an alternative to check against licensing. Apache Superset is released under the Apache License 2.0 and has no paid edition, while Metabase's enterprise directory uses a separate commercial license, so the choice should be confirmed against the distribution model before any build starts.

Customer-facing dashboards must never query operational networks. Under CIP-005-7, routable-connected assets of high and medium impact BES Cyber Systems must sit inside an Electronic Security Perimeter, and all Interactive Remote Access must pass through an Intermediate System with encryption and multi-factor authentication. A customer portal reading from a control-network historian would pull the portal's attack surface into that perimeter. The practical pattern is a one-way copy of meter and outage data from the operational side into a separate analytics zone, with Superset connecting only to that zone. Ask your compliance lead to map which BES Cyber Systems and historians feed the copy, because the impact classification under CIP-002-5.1a decides how tightly that feed is controlled.

Which requirements shape the build for utility dashboards

This table maps the regulatory and standards requirements most likely to shape an energy or utility dashboard build to what each one means for the architecture. Read it as a starting list for scoping: confirm each applicability question with your compliance lead and the registered entity responsible for your systems before design freezes.

Requirement	Where it comes from	What it means for the build
Impact classification of BES Cyber Systems	CIP-002-5.1a sorts BES Cyber Systems into high, medium and low impact using bright-line criteria	Confirm which systems feed or sit beside the dashboard before choosing hosting; a Superset or ClickHouse tier that touches in-scope systems changes the controls that apply
Electronic Security Perimeter and operator access	CIP-005-7 requires routable-connected assets of high and medium impact systems to sit inside an Electronic Security Perimeter	Keep operational data stores outside the dashboard boundary where possible and feed only approved aggregates across the perimeter
Interactive Remote Access with MFA	CIP-005-7 requires all Interactive Remote Access to pass through an Intermediate System with encryption and multi-factor authentication	Operator portal logins need an Intermediate System path and MFA, so plan SSO and the access gateway before building embedded views
Vendor and supply chain obligations	CIP-013-2 requires plans covering vendor incident notification, vulnerability disclosure, software integrity verification and reviewed at least every 15 calendar months	Document each open-source component (Superset, ClickHouse, reverse proxy, identity provider) with its source, release channel, patch owner and integrity checks
Information protection for dashboard data	CIP-011-3 covers protection of BES Cyber System Information	Classify what appears in dashboards and exports, restrict downloads and logs, and control who can query raw telemetry
Enforceable dates	NERC lists CIP-003-9 as enforceable 1 April 2026 and CIP-012-2 as enforceable 1 July 2026 (NERC CIP standards page)	Verify the standard versions in force for your entity on the build date and design logging and communications controls for them
Future standards for network monitoring and virtualization	CIP-015-1 is enforceable from 1 October 2028; virtualization revisions under FERC Order No. 919 take effect 1 July 2028	Design the dashboard stack so monitoring and virtualized hosting can be added without rebuilding the access model
EU energy cybersecurity classification	NIS2 Annex I lists energy, including electricity, oil, gas and hydrogen, as a high-criticality sector	EU operators should map NIS2 obligations to supplier security, incident reporting and access controls for the portal
Cross-border electricity flows	Commission Delegated Regulation (EU) 2024/1366 sets sector-specific cybersecurity rules for cross-border electricity flows	If dashboards expose cross-border flow data, confirm which network code rules apply to the data sources and hosting location

Which open-source platforms fit utility embedded analytics and why

Utility portals need a BI layer that embeds cleanly, a time-series store that handles meter volume, and an ingestion path that keeps operational data apart from customer views. The cards below cover the components that usually sit in that stack and the licensing or security checks each one needs before it reaches a customer or operator portal.

Apache Superset for embedded dashboards

Superset is licensed under Apache-2.0 and has no paid edition, so the licensing question is simpler than with dual-licensed tools. Its authentication options cover OAuth2 providers, custom OAuth2 servers using the code flow, and LDAP, with AUTH_ROLES_MAPPING translating directory groups into Superset roles. Check how customer and operator groups map to roles before any dashboard goes live. Run the metadata database on PostgreSQL or MySQL, because the SQLite default is not suited to production.

Metabase licensing for distributed portals

Metabase is licensed under the AGPL outside its enterprise directory, and enterprise code falls under the Metabase Commercial License. If a utility modifies Metabase and makes it available to users over a network, the AGPL obligations apply and need a legal reading before go-live. Confirm which features your plan requires, including embedding, SSO and per-user permissions, and whether they sit in the enterprise code. Ask counsel to review the distribution model before any customer sees a modified build.

ClickHouse for meter time-series

ClickHouse server is licensed under Apache-2.0 and fits high-volume interval data from smart meters, where queries filter by account, meter and time window. Replicated tables are coordinated by ClickHouse Keeper, and a recovered replica resynchronizes automatically. Replication does not protect against human error, so schedule BACKUP and RESTORE jobs and test restores on a spare cluster. Authentication can use LDAP, Kerberos, X.509 certificates or JWT, and the security policy lists which versions still get security updates.

Airflow or Dagster for meter file ingestion

Meter reads usually arrive as scheduled files or feeds that must be validated, deduplicated and loaded before dashboards refresh. Apache Airflow (Apache-2.0, 3.3.2 published September 17, 2026) and Dagster (Apache-2.0, 1.13.26 published October 9, 2026) both schedule and monitor these pipelines, so choose one based on your team's skills and existing tooling. Ingestion jobs should write to a store that separates operator-only tables from customer-facing views. Airbyte's platform repository is under the Elastic License 2.0, so review its terms before adopting it.

How customer views and operator views stay separated

Customers see usage and outage views for their own accounts, so the dashboard layer must enforce account scope on every query, not only on the page. The embedding design decides how that scope is carried. Guest tokens or SSO-based embedding can pass an account identifier into the session, and row-level security on the analytics database then restricts each query to the rows for that account. The application must treat the token as a credential for one account and nothing else.

Per-account row-level filtering: Apply row-level security on the meter and outage tables keyed by account or service point, so the filter lives in the database layer and not only in the dashboard definition.
Guest or SSO embedding: Confirm with the vendor how the embedding token is issued, how long it lasts and which claims it carries. Superset supports OAuth2 providers and custom OAuth2 servers using the code flow, and Metabase embedding is configured per product.
No cross-account queries: A customer token must never grant a query over another account. Test this with a second account in staging and try to alter filter parameters in the browser.
Separate customer datasets: Expose customer views through a curated dataset or view, not the raw operational tables, so a dashboard author cannot widen the query by accident.
Session and token expiry: Set short lifetimes and revoke tokens when a service agreement ends or an account is transferred.

Before launch, ask for a written test plan that covers account boundary tests and a list of every dataset a customer-facing dashboard can reach.

Why the CIP impact rating decides where analytics can run

Classify every system that touches the energy estate before choosing where the dashboard runs. Under CIP-002-5.1a, BES Cyber Systems are sorted into high, medium or low impact using bright-line criteria, and that rating decides which systems must sit inside an Electronic Security Perimeter under CIP-005-7 and which requirements apply to them. High impact mainly covers qualifying control centers. Medium impact includes, for example, generation of 1500 MW or more at one plant or transmission facilities at 500 kV or higher. An analytics platform that is hosted near these systems, or that shares a host, network segment or identity path with them, can pull the platform into scope even if its purpose is reporting. The impact determination belongs to the entity's own compliance program, so the first task is to obtain the current categorization for the assets that feed the outage and usage data.

A dashboard system that receives only exported, non-operational data may sit outside the BES Cyber System scope, but that conclusion must be confirmed, not assumed. Document the export path from the operational historian or meter data system to the analytics database, including the transfer method, who can initiate it and whether any return path exists. Document the identity system that authenticates staff and customer users, because a shared directory with operational accounts can change the analysis. Document any shared host, hypervisor or network segment, since shared infrastructure is a common reason a supposedly separate reporting system ends up in scope. Then take this package to the entity's compliance team and the relevant Regional Entity, and get written confirmation of the classification before the platform goes live. The NERC CIP standards page is the starting point for confirming which standard versions and effective dates currently apply.

What auditors and regulators ask for from a utility dashboard stack

This table maps the records an auditor or regulator is likely to request from an energy dashboard stack to the component that produces each one. Use it to plan evidence collection before an assessment, and confirm with the auditor which records satisfy each standard in your registration. The main takeaway is that most evidence comes from configuration, access and recovery records that must be kept current, not from the dashboards themselves.

Evidence	What it shows	Component that produces it
Asset and impact classification record	Which systems are BES Cyber Systems and whether each is high, medium or low impact under CIP-002-5.1a	Your asset inventory and CIP-002 program, with the dashboard stack listed as a dependency
Electronic Security Perimeter and data flow diagrams	How routable assets sit inside the ESP under CIP-005-7 and where operational data crosses into the customer-facing portal	Network architecture documents, firewall rules and the Intermediate System for interactive remote access
Superset role mapping configuration	Which LDAP or OAuth groups map to which Superset roles through AUTH_ROLES_MAPPING, showing who can see operational versus customer views	Superset configuration file and identity provider group exports
ClickHouse access control and external authenticator configuration	Which users, roles and row policies can read time-series tables, and which directory or certificate authenticates them	ClickHouse access control definitions and external authenticator settings (LDAP, Kerberos, X.509 or JWT)
Backup and restore test records	That backups exist, that restores were tested on a spare cluster and that results were recorded, as the ClickHouse backup guidance advises	ClickHouse BACKUP and RESTORE job logs plus signed restore test reports kept by the operator
Keeper and replication status records	Health of the three or more ClickHouse Keeper nodes and that replicas resynchronized after a failure	ClickHouse Keeper and system replication status outputs, retained with dated snapshots
Upgrade records with staging tests	That each Superset or ClickHouse upgrade was backed up, tested in staging and reviewed against UPDATING.md or the changelog	Change tickets, staging test results and the release notes reviewed for each version
Supplier and component version inventory	Software components, versions and their suppliers, supporting CIP-013-2 vendor risk and software integrity reviews	Software bill of materials generated during build, plus vendor notification and vulnerability disclosure contacts

Rollout for embedded analytics for energy and utilities portals

The sequence moves from classifying the systems in scope to a staged launch in the portal, with each phase ending in a named output that the operator can check before the next phase starts. Durations come from the written scope agreed before work begins, so this page does not set timelines.

Confirm scope and regulatory classification
Before any build, DeployOpen works with the operator's compliance and OT teams to identify which systems touch the bulk electric system, which assets are routable-connected, and whether the entity is registered under the NERC CIP Reliability Standards, subject to NIS2 as an energy entity, or both. The classification decides which control families apply to the dashboard stack, so the operator should confirm its registration status with its Regional Entity or national authority rather than assume it. The output is a signed scope document listing in-scope systems, applicable standards by ID, and the owner of each control. The step is done when that document is approved in writing.
Decide the data flow and trust boundary
The team maps every path that meter and outage data takes from the operational network to the customer portal and the operator console. The key decision is where the boundary sits between operational data and customer-facing views, and whether the analytics database is inside the same perimeter as operational systems or in a separate zone reached only through a controlled one-way or tightly filtered path. Dashboards for customers should never query operational historians or control-network systems directly. The output is a data flow diagram with trust zones, the list of allowed connections and the authentication used on each one, reviewed by the security architect.
Build the ingestion pipeline and ClickHouse schema
DeployOpen builds the ingestion path from meter head-end or data collection systems into ClickHouse, with separate tables for interval readings and for outage events, keyed by meter or account and by time. Replicated tables are set up with ClickHouse Keeper, which the replication guide recommends running as at least three nodes on dedicated hosts in production. Backups use BACKUP and RESTORE to storage the operator controls, and the team confirms that a recovered replica resyncs from a healthy peer. The output is a deployed schema, a tested load path and a documented backup job, with the version chosen from the currently supported ClickHouse release lines.
Deploy Superset on a production metadata database
The team installs Apache Superset with PostgreSQL or MySQL as the metadata database, using a supported version from the configuration documentation, and does not use the SQLite default or the Docker Compose PostgreSQL container for production. A user-specified SECRET_KEY is set before first start, and the caching layer and Celery workers are deployed where alerts, reports or async queries are required. Dashboards for customers are restricted to a read-only analytics connection. The output is a running staging environment with documented configuration keys, secrets held in the operator's vault, and a checked upgrade path that begins with a metadata database backup.
Configure SSO, role mapping and row-level security
DeployOpen connects Superset to the operator's identity provider through OAuth2 or LDAP, then maps groups to Superset roles with AUTH_ROLES_MAPPING so that customer users and operator staff receive different access. Row-level security rules tie each customer login to the accounts that belong to that customer, and the rules are applied on the database side wherever possible. The output is a role matrix approved by the operator and a configuration that can be reviewed by an auditor. Because mapping errors are the most common source of cross-customer exposure, this step closes only after the tests in the next phase pass.
Test row-level security with test accounts
The team creates customer and operator test accounts for several illustrative service territories and runs scripted checks that attempt to reach another customer's meter intervals, outage records or operator-only fields through the portal, the API and direct queries. Each test records the expected result, the actual result and the evidence. Any failure is fixed and the full suite is run again. The output is a signed test report that lists each case and its result, which the operator can keep as evidence for CIP or NIS2 assessments where relevant.
Run security testing and a restore test
DeployOpen runs application, API and infrastructure security tests against the staging stack, covering the portal integration, the Superset deployment and the ClickHouse endpoints, with findings triaged by severity and fixed before launch. Separately, the team restores the latest backup to a spare cluster and checks row counts, sample queries and dashboard output against production. The ClickHouse backup documentation warns that an untested backup will probably not restore when needed, so this restore is a required gate. The output is a test report and a restore record that shows the date, the backup used and the match results.
Launch in stages and hand over runbooks
The embedded dashboards are released first to a limited group of customer and operator users, then widened once monitoring shows query load and access logs match the approved role matrix. DeployOpen then hands over runbooks for upgrades (including the Superset upgrade steps and reading the release notes before each version change), backup and restore, key rotation, access reviews and incident response, and trains the operator's staff on them. The output is a launch record, the signed runbook set and a named support owner on each side. Ongoing support and upgrades can be covered by a retainer if the operator wants it.

What embedded analytics for energy and utilities costs and requires

What drives the cost of embedded analytics for a utility portal?

Cost follows the components you need to run, and most of them recur after launch. The main lines are the analytics database that stores smart meter time-series, the Superset components (the application, a metadata database, and for alerts, reports and async queries, a caching layer and Celery workers), the ingestion pipeline from meter data systems, security testing, compliance evidence work, and ongoing upgrades and backups. Superset is Apache-2.0 licensed with no paid edition, so license fees are not the main driver. Infrastructure, integration and operating effort are. Ask any partner for a quote that separates one-time build work from recurring support, upgrades and retainers, and that names who pays for each line. DeployOpen does not publish prices, so the figures come from the written scope.

How long does an embedded analytics build for a utility take?

The timeline depends on the scope, the number of data sources, how many portal views are needed and how quickly classification and access decisions are made on the utility side. DeployOpen does not publish a fixed duration. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts, so the answer should come from that document. Ask for the phases, the acceptance criteria for each one, and which items depend on the utility's own approvals, such as change windows and identity system access. Time spent on a staging environment and a restore test also belongs in the plan.

What does the utility's team need to provide before the build starts?

The utility team supplies the decisions and access that the build cannot make alone. Expect to provide:

Classification records for the systems that feed the dashboards, so the team knows which data is operational and which can reach a customer-facing view
Access to the identity system, so that SSO groups or OAuth claims can be mapped to dashboard roles
Representative meter data samples, with volumes and interval granularity, so query performance can be tested
Change approvals for staging and production changes, including backup and restore windows

Missing any of these usually moves work to the right, so confirm owners and dates in the scope before kickoff.

Can a utility run embedded analytics in-house without outside help?

A capable internal team can run it, but the skills must cover more than dashboard design. The stack needs people who can operate Superset with PostgreSQL or MySQL as its metadata database, configure its SECRET_KEY and authentication, manage the caching and worker components, and run upgrades following the Superset upgrade steps: back up the metadata database, run the database upgrade and init commands, test in staging and check the breaking-change notes. The analytics database also needs operators for replication, Keeper nodes, backups and restore tests. Assess whether your team has these skills, and whether it can staff them through upgrades and on-call periods. A partner can cover part of this through a fixed-scope build and a later retainer.

Can an open-source analytics stack satisfy NERC CIP audits?

No single product satisfies CIP. There is no CIP certificate. Entities registered as a Balancing Authority, Reliability Coordinator or Transmission Operator receive a compliance audit at least once every three years, and the regional entity and NERC decide the outcome. DeployOpen prepares systems, controls and evidence for that assessment, such as network placement, access records, change logs and backup evidence, but it does not issue audit reports or attestations. Whether a dashboard system falls under CIP at all depends on the BES Cyber System classification of the systems it touches, which the utility determines under CIP-002-5.1a. Ask your compliance lead to confirm that classification before design begins.

Is Metabase or Superset safer for a customer-facing utility portal?

Apache Superset is the safer default for most utility portals because it is licensed under Apache-2.0 with no paid edition, and its authentication options include OAuth2 providers, custom OAuth2 servers using the code flow, and LDAP, with group-to-role mapping through AUTH_ROLES_MAPPING. Metabase is AGPL licensed outside its enterprise directory, which uses the Metabase Commercial License, so review how the AGPL and the commercial terms apply to your distribution and which features need a paid plan. Safety depends more on the design than the tool: row-level security per account, a separate analytics store for customer views, and no direct access from the portal to operational systems. Confirm each point in a proof of concept.

What does the security review cover for a utility dashboard?

The review should follow the operational technology and network segmentation questions that the CIP standards raise. Check whether the dashboard system sits outside any Electronic Security Perimeter that protects high or medium impact BES Cyber Systems, and whether any remote access passes through controlled, encrypted and multi-factor authenticated paths as CIP-005-7 requires. Review supply chain risk for each open-source component under CIP-013-2, including software integrity verification and vendor remote access. NIST SP 800-82 Rev. 3 and the ISA/IEC 62443 series are useful reference points for operational technology security. DeployOpen runs security testing of applications, APIs and infrastructure as a separate service, and the test scope should be set in writing.

What to ask an implementation partner about utility embedded analytics

Start with classification. Ask the partner how it assigns each system in the design to a CIP impact level under CIP-002-5.1a, and whether it will document the Electronic Security Perimeter boundary before any build starts. Ask which components sit inside that perimeter (the analytics database such as ClickHouse, the Superset application and metadata database, caching and workers) and which sit outside it, such as the customer portal that calls the embedded dashboards. Request the written boundary diagram and the list of routable connections, and confirm how Interactive Remote Access to the operator side passes through an Intermediate System with encryption and multi-factor authentication under CIP-005-7. Ask who on the partner side will sign that diagram and who on yours will approve it.

Next, ask how row-level security is tested rather than assumed. Ask for a test matrix that covers each customer account and each operator role, with the expected result for every combination, including attempts to read another account through a URL, a saved chart, a dataset query or an export. Ask how the partner maps LDAP or OAuth groups to Superset roles with AUTH_ROLES_MAPPING and how a role change is removed from active sessions. Also ask how the partner keeps operational outage data separate from customer usage views, and whether it will run those tests again after each upgrade. Finally, ask how backups are restore-tested: the ClickHouse backup documentation warns that an untested backup will probably not restore when needed, so ask for restore test results on a spare cluster, not only backup job logs.

Then ask about the paper trail. For CIP-013-2, ask how each open-source component (Superset, the ClickHouse server, Keeper, the reverse proxy and their container images) is recorded in a bill of materials, how vendor vulnerability notices are received and tracked, and how software integrity is verified at install and upgrade. Ask how the partner handles license terms: Superset is Apache-2.0 with no paid edition, but Metabase is AGPL outside its enterprise directory and some components carry Elastic License terms. Ask who reviews those terms and what happens if the design later needs a component with restrictions. On scope, ask who signs the fixed-scope agreement, what the written deliverables list (architecture, configurations, test evidence, runbooks, handover), and what a retainer covers, especially Superset major-version changes that require a checked UPDATING.md review and ClickHouse support windows for the version in use.

How DeployOpen scopes and builds utility dashboards

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source analytics inside the client's own infrastructure. For energy and utilities embedded analytics, we start from a written scope that separates customer-facing usage views from operator outage data, then build and hand over the platform in the client's own cloud account, data center or air-gapped network.

How we scope

Written scope lists which customer, operator and utility staff roles see which dashboards and datasets.
Scope records meter data sources, refresh expectations and the per-account filter model for each portal.
Scope names the identity provider, SSO method and group-to-role mapping that govern portal access.
Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.

How we build

We deploy Apache Superset with a production metadata database, not the SQLite default used for quick installs.
Row-level security rules are built and tested per account so customers see only their own meter data.
Operational outage data is kept in a separate dataset and workspace from customer usage views.
Caching, alerts and worker components are configured where dashboards depend on them, with the SECRET_KEY set explicitly.

How we hand over

Handover includes architecture, configuration, access model and row-level security test records.
Runbooks cover backup of the metadata database, staged upgrades and restore testing for the analytics store.
Documentation maps access controls and logging to the evidence your compliance reviewers will request.
Clients receive the platform in their own environment, with licensing terms for distribution reviewed in writing.

How we support

Ongoing support, upgrades and continuous security testing are available as retainers.
Upgrades follow a staging-first process with checks for breaking changes before production rollout.
Security testing can cover the portal, APIs and access controls that expose utility dashboards.
Support scope is agreed in writing, including who responds to incidents and how changes are approved.

Embedded analytics for energy and utilities