Apache Superset and Apache Pinot integration

Apache Superset and Apache Pinot integration: what the connection does

A team that needs low-latency dashboards on Apache Pinot can use Superset as the front end, and the supported way to do it is the pinotdb SQLAlchemy driver. Superset connects with a `pinot+ connection string, where the broker host and port name the component that receives queries and the controller URL names the component that manages cluster metadata. The default broker port listed in the Superset database catalog is 8099. Superset does not read Pinot segments or talk to servers directly. It sends SQL to the broker's query interface, and the broker routes the work to the servers that hold the relevant segments and returns the merged result to Superset.

This split matters for planning. The controller URL is used for metadata lookups, such as discovering tables for the dataset picker, while the broker carries the dashboard query load. Dashboards with many charts multiply the number of broker calls, so the broker tier is where capacity and concurrency limits show up first. Confirm with your Pinot operators which broker pool serves analytical traffic, whether brokers are reachable from the Superset workers and web nodes on the network path you plan, and whether TLS terminates at the broker or at a proxy in front of it. Also confirm that the pinotdb version in your Superset image supports the Pinot release you run, since the Superset driver and the Pinot server version are tracked separately. Pinot's own release notes, listed on its releases page, are the place to check that compatibility.

What you need before connecting Superset to Pinot

Connecting Superset to Pinot depends on three things being in place before any dashboard is built: a supported Superset release with a production metadata database, a Pinot driver in the Superset image, and a Pinot cluster that Superset can reach over the broker and controller endpoints. The cards below cover each of these, plus the version check that should happen first.

Superset version and release line

Choose the Superset release line before anything else, because the image, the driver set and the upgrade path all follow from it. The current stable line is 6.1.0, published on May 13, 2026, and 7.0.0 is still a release candidate. Pin the exact version in your image build and record it alongside the driver version so that a later upgrade can be tested against a known baseline. Confirm the release you intend to run against the Superset releases page before the build starts.

Production metadata database

Superset stores dashboards, users, roles and chart definitions in a metadata database, and the default SQLite database used by quick installs is not suitable for production. Plan for PostgreSQL 10 to 17 or MySQL 5.7 and 8.x, and keep this database separate from the Pinot cluster. Its backup and restore procedure should be tested before the Pinot connection goes live, since dashboard definitions are lost without it.

Pinot driver in the Superset image

Superset reaches Pinot through the pinotdb Python package, which is not part of a default Superset image. Add it to the image build used for your web and worker containers, so that every process that runs queries has the same driver. Record the pinotdb version in your image manifest and confirm compatibility with your Superset release, since driver requirements change between Superset versions. Test the connection from inside the running container, not only from your laptop.

Pinot version and JDK 25 for services

Pinot services require JDK 25 or later to build and run, while Java and JDBC clients still target Java 11 bytecode. This matters for the teams that operate the Pinot cluster, not for Superset itself, but it affects image builds and host baselines for controllers, brokers, servers and minions. Confirm the Pinot version your cluster runs against the Pinot releases page, since the 1.5.1 and 1.6.0 release candidate are on different dates and the query behavior you test should match the version in production.

Broker and controller network paths

The Superset connection string uses the broker host and port, which default to 8099, and a controller URL that Superset uses to resolve table metadata. Both endpoints must be reachable from every Superset web and worker container, which usually means a firewall or security group rule from the Superset subnet and, for Kubernetes, a network policy. Ask the Pinot team which broker and controller addresses are exposed to application tiers and whether they sit behind a load balancer.

Access identity for the Pinot connection

Decide which Pinot identity Superset uses before the connection is created. A shared service account gives every dashboard viewer the same data access, so row-level scoping and dashboard permissions must be designed against that account. Ask the Pinot operators whether broker or controller authentication is enabled in your cluster, and whether the account can be limited to the tables that dashboards need. Store any credentials in Superset's configured secret handling, not in the connection string shown in the UI.

How to configure the pinotdb connection step by step

This sequence takes a self-hosted Superset from a running instance with no Pinot connection to a saved database entry that answers a test query in SQL Lab. Each step ends with a check you can run before moving on.

Confirm Superset and Pinot network reachability
Before touching Superset, the platform team confirms that the Superset web and worker hosts can open TCP connections to the Pinot broker port and the Pinot controller port in your environment. Run the test from the same host or container image that will run Superset, not from a laptop, because firewall and DNS rules often differ between them. You know this step is done when both host and port pairs accept connections from the Superset runtime and the controller address resolves by the name you plan to use.
Install the pinotdb driver into the Superset environment
The Superset administrator adds the pinotdb package to the Python environment that runs Superset, which is the container image for container installs or the virtual environment for PyPI installs. Driver installation is a build-time change, so it belongs in the image or dependency file that your deployment process rebuilds, not in a manual change inside a running container. Confirm the install by importing the package in the same interpreter that serves Superset and by restarting the web and worker processes so they load it.
Build the pinot+http connection string
Write the SQLAlchemy URI in the form pinot+ The broker port defaults to 8099 in the Superset database catalog, but confirm the value your Pinot cluster actually exposes. Replace each placeholder with the internal DNS name or address you verified in the first step, and keep the controller URL's trailing slash as shown in the catalog format. Store the final string in your secrets manager rather than in a shared notes file, since the URI can carry credentials when your Pinot deployment requires them.
Add the Pinot database in Superset
In Superset, an administrator opens the database connections page, chooses to add a database, selects Apache Pinot from the supported list, and pastes the connection string. Choose a display name that tells analysts which cluster and environment they are querying, such as pinot-prod or pinot-staging. Leave advanced options such as query limits and caching for a later review, and save the connection. You know this step worked when the database appears in the connections list without a validation error.
Run the connection test
Use the test connection action on the saved database form before you finalize it. A successful test shows that Superset reached the broker, that the broker answered a metadata request, and that the controller URL was accepted. If the test fails, read the error message against the placeholders in your string, then check the broker port, the controller port and whether the Superset runtime can reach both. Repeat the test until it passes, then record the Superset version, pinotdb version and Pinot cluster name in your change record.
Verify with a sample query in SQL Lab
Open SQL Lab, select the new Pinot database, and run a short query against a table you know exists, such as a SELECT with a LIMIT clause that returns a few rows. Confirm that the column names, types and row counts match what the Pinot team reports for that table. Keep the first query small so you do not load the cluster during setup. You know the connection is usable when the query returns rows and the result matches your expectation for that table.
Confirm the access and load boundaries before sharing
Before analysts use the connection, the platform team confirms which Superset roles can see the Pinot database and whether the service account used in the connection string has read-only access to the tables it needs. Ask the Pinot administrators which tables and tenants this account can read, and set expectations for query limits with them, since dashboards issue queries continuously. The step is complete when the role list is recorded and a second, non-administrative user can run the same sample query.

Why Kafka ingestion decides how fresh Superset dashboards are

Dashboard freshness is set by the Pinot ingestion path, not by Superset. Superset sends queries to the Pinot broker and renders whatever the broker returns, so a chart can be no newer than the data Pinot has loaded into its segments. Real-time tables ingest from Apache Kafka, Apache Pulsar and AWS Kinesis, while offline tables load batch files such as CSV, Avro or Parquet. Pinot can combine batch and streaming sources into a single table, which lets a dashboard show historical data and recent events side by side. Before any chart is built, the team should agree which of these paths feeds each table and what latency the business expects from it.

For the Kafka path, confirm the details with the Pinot team and the platform owner of the Kafka cluster before building charts. Ask which topics feed each real-time table, how many partitions each topic has, and whether message keys keep related events on the same partition. Ask whether upserts are enabled, which primary key the upsert uses, and how late or duplicate events are handled. Also confirm the retention period on each topic, since a replay or backfill after an outage depends on it. These answers determine whether a chart reflects the latest state of an entity or every event ever emitted, and a dashboard built on the wrong assumption will look correct while reporting the wrong number.

Treat the batch and streaming mix as a design decision with an owner. Document which dashboards need seconds-level freshness, which need hourly loads and which can read yesterday's offline segments, then ask the Pinot team how segments are built and pushed for each table and how often. Verify freshness on a staging cluster by publishing known test events to Kafka and checking when they appear in a Superset chart. Record that measured lag next to each dashboard so users and auditors can see what the data represents.

How Pinot tables, columns and Superset datasets map

This table shows how each Pinot object and connection setting maps to a Superset object or field, and where the mapping needs confirmation. Read it as a checklist for the design session, not as a fixed configuration. The main takeaway is that the Pinot schema and the Superset role design must be confirmed in your environment before dashboards go to users.

Pinot or Superset item	Mapping in Superset	What to confirm in your environment
Pinot table (real-time or offline)	One Superset dataset per queryable table or SQL view	Whether real-time and offline tables are combined, and which name users will see
Pinot time column	Temporal column used for time grains and time range filters	The time unit and format in the schema, so grains such as day or hour compute correctly
Pinot columns (dimensions and metrics)	Dataset columns and metrics defined in Superset	Which columns are dimensions and which are aggregates, and that metric expressions match Pinot SQL support
Broker host and port (default 8099)	Host and port in the pinot+http connection string	That the broker is reachable from the Superset web and worker nodes, not only from your workstation
Controller URL	Value of the controller parameter in the connection string	That the controller address resolves from Superset and that it matches the cluster you intend to query
Superset database connection	One database entry per Pinot cluster or environment	Separate connections for development, staging and production so dashboards do not cross environments
Superset roles and data access	Roles grant access to databases, datasets and dashboards	That role design maps to who may see which Pinot tables, and that no broad role exposes sensitive columns
Pinot query limits and multi-stage queries	Superset SQL Lab and chart queries pass through the broker	Which queries need limits in Superset and which Pinot query options your team will use for heavy joins

Query limits and load controls for user-facing and operational dashboards

Dashboards shown to many external or internal viewers at once create bursts of identical or near-identical queries. The load pattern is driven by how many people open a page at the same time, not by how often one analyst refreshes a chart. Start by estimating peak concurrent viewers with the product owner, then decide which charts must be live and which can tolerate cached results. Pinot is well suited to this pattern when queries are narrow and indexed, but a poorly designed dashboard can still saturate brokers and servers.

Cache first: Use Superset's caching layer (typically Redis in most installations) so repeated chart requests for the same filters do not reach Pinot. Set cache timeouts per dataset based on how fresh the data must be.
Restrict chart types: Prefer aggregated tables and time series over raw row tables with large limits, since wide result sets increase broker merge work.
Use async queries deliberately: Async queries depend on the worker components described in the Superset architecture documentation. Confirm the worker and caching setup before relying on them for viewer-facing pages.
Limit filter fan-out: Dashboards with many cross-filters multiply query count per page load. Review filter counts with the design team.
Confirm broker capacity: Ask your Pinot operator how many brokers serve the pinot+http URL and how traffic is distributed across them.

Before launch, run a load test that mirrors the expected peak concurrency and filter mix, and check the result against your own latency targets.

The Superset connection string sends queries to a broker, and the broker decides how each query fans out to servers. The controller URL in the same connection string is used for metadata, not for query execution. Understanding this split helps you find the true bottleneck when latency rises. Brokers route queries and merge results, servers host segments and process queries, and the broker is the single entry point Superset sees. Pinot's architecture documentation describes how the broker computes a plan for the multi-stage query engine that runs across several sets of servers and returns the final stage's result to the client.

Broker URL: The default broker port in the Superset catalog is 8099. Confirm the host and port your Pinot operator exposes, and whether a load balancer sits in front of several brokers.
Controller URL: The controller parameter points Superset at the cluster controllers, which manage cluster state and node failures. Confirm that the Superset host can reach it.
Single-stage versus multi-stage: Queries with joins or complex operators can take the multi-stage path. Ask the Pinot team which query shapes your dashboards generate and test them explicitly.
Broker sizing: Brokers hold merge work for every query. Confirm how many brokers the cluster runs and whether they are dedicated to Superset.
Query logging: Enable broker-side query logging where your Pinot distribution supports it, so you can match slow Superset charts to specific queries.

Ask your Pinot team to confirm the current multi-stage engine settings for your release, since defaults and options change between versions.

Securing the Superset to Pinot path

Superset sends queries to Pinot brokers and reads connection details from its own metadata database, so the path has several places where access can leak. The cards below cover the controls to set on the Superset side, the Pinot endpoints it reaches, and the people who can change the connection.

Read-only Pinot query access

Superset only needs to run SELECT queries against Pinot tables, so the connection should not be the path for any administrative or write action. Pinot's SQL interface is served through the broker, and Pinot does not give Superset a separate database login to lock down in the same way a relational source does. Confirm with your Pinot team which authentication options your cluster version supports for broker access, and make the Superset connection use the narrowest identity that option allows.

Restrict broker and controller ports

The connection string points at a broker port, with the default broker port 8099, and a controller URL that Superset uses to resolve table metadata. Allow traffic only from the Superset web and worker hosts to those ports, and keep server and ZooKeeper ports closed to Superset entirely. Document the firewall rules as part of the connection record so a network change does not silently break dashboards or widen exposure.

TLS on broker and controller endpoints

Where your Pinot deployment terminates TLS on the broker and controller endpoints, Superset's connection should use HTTPS and verify the certificate chain against your internal CA. Check the pinotdb driver's options for certificate verification in the Superset version you run, and do not disable verification to get a connection working. Plaintext HTTP between Superset and Pinot is acceptable only on a segregated network you have documented and accepted.

Keep the connection secret out of plain view

The Pinot connection string can carry credentials or tokens, and it is stored in Superset's metadata database. Store secret values in environment-backed configuration or a secrets manager rather than typing them into the connection form, and restrict who can read the database record. Plan rotation for any token used, and confirm how your Superset version masks sensitive fields in the UI and API responses.

Protect the Superset SECRET_KEY

Superset requires a user-specified SECRET_KEY to start, a requirement added in version 2.1.0, and it signs sessions and encrypts stored connection secrets. Generate a long random value per environment, keep it out of source control, and use PREVIOUS_SECRET_KEY when you rotate it so existing sessions and stored credentials remain readable during the change. Losing this key can make stored Pinot credentials unrecoverable, so back it up with your metadata database.

Map roles and limit connection editors

Superset's role model decides who can create, edit and delete database connections, and that permission is the most sensitive one on this path because it lets a user point Superset at any broker. Use AUTH_ROLES_MAPPING to map identity provider groups to Superset roles, keep connection-editing rights with a small platform group, and give analysts dashboard and SQL Lab access only. Review the membership of that group on a schedule and log changes to connection records.

Common Pinot connection failures and how to fix them

This table lists the failures teams most often hit when connecting Superset to Apache Pinot, with the likely cause and a general fix tied to the broker and controller settings. Work down the rows in order, since a wrong broker port or unreachable controller blocks every later check. Confirm each fix against your own cluster before changing production settings.

Failure	Likely cause	How to fix it
Connection refused or timeout on the broker	The broker_port in the pinot+http connection string does not match the broker's listening port. The documented default broker port is 8099, but your cluster may differ.	Confirm the broker host and port from your Pinot deployment, then update the SQLAlchemy URI to pinot+
Query works but table or schema lookups fail	The controller parameter is missing or points to a host Superset cannot reach. Metadata calls go to the controller, not the broker.	Check that the controller URL resolves and answers from the Superset web and worker containers, not only from your laptop. Open the controller port on the network path Superset uses.
Driver not found when adding the database	The pinotdb package is not installed in the Superset image or virtual environment. Installing it on one host does not cover the others.	Add pinotdb to the image or environment that runs the web server and Celery workers, rebuild, and restart all Superset processes. Verify with a test connection from the database settings page.
Connection errors after an upgrade or on a new host	Python dependencies or the Pinot version drifted between environments. Pinot services require JDK 25 or later to build and run, and client compatibility should be checked against the Pinot release you run.	Pin the pinotdb version in your build, confirm the Pinot cluster version from the releases page, and run a test query in staging before promoting. Ask your platform team to confirm the JDK on Pinot nodes.
Queries slow or failing on large tables	Filters or group-by columns lack suitable indexes, so servers scan more segments than necessary. Pinot offers several index types, and the right choice depends on query patterns.	Review the SQL that Superset generates for slow charts, then confirm with the Pinot team which columns need inverted, range, or other indexes. Test index changes on a copy of the table before applying them.
Dashboards time out under load	Many users or auto-refreshing charts send concurrent queries to the broker. Superset-side settings such as query timeouts and caching also determine how long requests hold on.	Reduce refresh frequency, enable caching for shared dashboards, and set sensible query timeouts in Superset. Ask your Pinot team to size brokers and servers against the expected concurrency and to review query limits.
Errors on multi-stage or join queries	The query uses features that depend on the multi-stage query engine, and the broker plan or server capacity does not handle the request.	Check whether the failing query uses joins or features that need the multi-stage engine. Simplify the chart query or pre-aggregate in a Pinot table, then confirm behavior with the Pinot team.

Questions buyers ask before connecting Superset to Pinot

What drives the cost of connecting Superset to Apache Pinot?

The main cost driver is how much of the Pinot side you need built, not the Superset connection itself. Pointing Superset at an existing Pinot broker is a small task, while the larger work includes Pinot table and schema design, Kafka ingestion, index choices, access design and the Superset metadata database, caching and worker setup. Costs split between one-time work (design, build, validation, handover) and recurring work (upgrades, monitoring, dashboard changes). Ask any quote to itemize these parts, to state which Pinot and Kafka components are in scope, and to name who operates each one after handover. Pinot's own design choices, such as whether you ingest from Kafka in real time or load batch files, change the effort more than the dashboard count does.

How long does a Superset and Pinot integration take?

Duration depends on scope, and no fixed timeline applies across environments. A narrow scope, such as one existing Pinot cluster, one broker endpoint and a few dashboards on pre-built tables, moves faster than a project that adds new real-time tables from Kafka, multi-stage queries and production Superset hardening. Pinot has several moving parts (controllers, brokers, servers, minions, ZooKeeper and the deep store), and access requests to each can add elapsed time even when the engineering work is short. Ask for a written schedule with phases, the dependencies on your teams and the acceptance tests for each phase.

What must our team provide before the work starts?

Plan to provide the following before the engagement starts:

Network and access to the Pinot controller and broker endpoints, plus a test environment that matches production topology.
Table and schema definitions, or the business questions the dashboards must answer, so the team can review them.
Kafka details for real-time tables: topic names, message format, partition counts, retention and the consumer access you approve.
Identity design: which users, groups and roles may query which tables, and how Superset roles map to those permissions.
Named owners for Superset, Pinot and Kafka who can approve changes and test results.

Confirm each item with your platform owners in writing, because missing access is one of the most common delays.

Can our internal team connect Superset to Pinot without a partner?

Yes, a capable internal team can make the basic connection, since the Superset database catalog lists the pinotdb package and a connection string built from a broker host, broker port (default 8099) and controller URL. The harder part is the work around that connection: designing Pinot tables for query patterns, choosing indexes, sizing servers, setting query limits, and keeping real-time ingestion from Kafka healthy. Pinot also runs on a Java runtime that the README says requires JDK 25 or later, so the team needs the skills to run that stack. Decide honestly whether your team has Pinot and Kafka operations experience. If not, a partner can cover design, build and handover while your staff learn to run it.

What recurring work follows after the integration goes live?

Recurring work includes Superset upgrades, Pinot upgrades, Kafka consumer monitoring, index and table changes as query patterns shift, and access reviews as people change roles. Superset's upgrade guidance asks you to back up the metadata database, run the upgrade commands, test in staging first and check UPDATING.md for breaking changes. Pinot releases are also frequent, and the apache/pinot releases page lists 1.5.0 on April 9, 2026 and 1.5.1 on June 5, 2026, so plan a regular cadence rather than ad hoc patching. Ask any quote whether upgrades, monitoring and on-call support are included, how often they are reviewed, and whether they fall under a retainer.

What are the main risks when Superset queries Pinot directly?

The main risks are overloading brokers or servers with dashboard traffic, exposing data to users who should not see it, and running unbounded queries against real-time tables. Superset dashboards can refresh often and many users can open the same chart at once, so query limits, caching and dashboard design matter as much as the connection. Access risk comes from giving the Superset service account broad read access to every table. Ask for query limit settings, a read-only Pinot identity, a documented role mapping and a load test that uses realistic dashboard concurrency before go-live.

Which Pinot and Superset versions should we confirm before we start?

Confirm versions with your vendor and in the source repositories before the design is final, because Superset and Pinot both publish frequent releases. The Superset project reports 6.0.0 published December 18, 2025 and 6.1.0 published May 13, 2026 on PyPI, and a 7.0.0 release candidate appears on its releases page. For Pinot, the releases page lists 1.5.1 on June 5, 2026 and a 1.6.0 release candidate on October 17, 2026. Ask the quote to name the exact versions in scope, the driver version that works with your Superset release, and the Java runtime for Pinot services, and to state how version changes after the start date are handled.

Does DeployOpen run the Pinot and Superset environment for us?

DeployOpen plans, implements, customizes, integrates, hardens, backs up, upgrades, documents and supports self-hosted open-source platforms inside your own infrastructure, including your cloud accounts, data centers or air-gapped networks. For this integration, that can cover the Superset connection, Pinot table and access design, Kafka ingestion setup, documentation and handover. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts, and ongoing support or upgrades can be arranged as a retainer. DeployOpen does not publish a price list, so request a quote built from a written scope.

How DeployOpen scopes and delivers the Superset and Pinot integration

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure. For Apache Superset and Apache Pinot integration, we agree the scope in writing, build the connection and dashboards in your environment, and hand over the configuration and runbooks your team needs to operate it.

How we scope

Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
The written scope lists which Pinot broker and controller endpoints, Superset roles and dashboards are included.
We confirm your Pinot version, Kafka ingestion design and the network path from Superset to the Pinot brokers.

How we build

We configure the pinotdb connection string with your broker and controller hosts and verify it against your cluster.
We set query limits and test dashboard queries against realistic table sizes before users depend on them.
We review how real-time Kafka tables and offline tables are queried, so dashboards reflect the data your team expects.

How we hand over

You receive the connection configuration, role mappings and dashboard inventory documented against the agreed scope.
We provide runbooks covering connection changes, Pinot version upgrades and troubleshooting slow dashboard queries.
Your team walks through the deployment and the test results so operators can maintain the integration without us.

How we support

Ongoing support, upgrades and continuous testing are available as retainers for the Superset and Pinot integration.
Retainers can cover checking Superset and Pinot release notes before upgrades, including the UPDATING.md review.
Support work is scoped in writing, so changes to dashboards, connections or Kafka ingestion are priced from a defined baseline.

Apache Superset and Apache Pinot integration