Open source data lakehouse RFP questions

Open source data lakehouse RFP questions to ask before you choose a partner

A partner proposing a lakehouse build should be able to design the catalog and governance, build the pipelines and query layer, run the platform after go-live and hand it over cleanly. The questions below test those four abilities, and they are written to be neutral to vendors: no particular table format, engine or catalog is assumed to be the right answer, so each partner has to justify its own choices. Send one written scope request to every shortlisted partner with the same data volumes, freshness targets, source systems, user counts and compliance obligations, and ask for the answers in writing against the same criteria. Score each answer on the same scale, record who answered each question, and ask for a live demonstration of the risky parts (a catalog-backed table being created, a time travel query, a failed compaction being recovered) rather than accepting slides as evidence.

A useful way to frame the scoring is to ask every partner the same core question: which decision is the hardest one for your design, and what would make you reverse it? A strong answer names concrete trade-offs, such as how the chosen table format is read by the engines the business already uses, or how the catalog is backed up and recovered. A weak answer lists product features without linking them to the data estate. Ask each partner to confirm the versions it intends to deploy, and check them against the vendor's own release notes or repository rather than the proposal. Where a partner cites a licence, ask for the specific licence file and the clauses that apply to your intended use, since licences that restrict hosted or commercial use can change the architecture and the cost.

Which questions test catalog and governance design?

Which table catalog should our open lakehouse use, and why?

The right catalog depends on the engines you run and the systems you already operate, so a good answer names one catalog and explains the trade-offs against the others. Trino's Iceberg connector can use a Hive metastore, AWS Glue, JDBC, a REST catalog, Nessie or Snowflake as the catalog, selected through the iceberg.catalog.type setting. Ask the partner to state which catalog it recommends, which engines must reach it (Trino, Spark, ClickHouse or others), where its metadata database runs, and how it handles catalog high availability. A weak answer names a catalog without tying it to your engines or your existing identity and network controls. Ask for the reasoning in writing so it can be reviewed against your requirements.

How should access control work across Trino, Spark and the catalog?

A good answer states where access decisions are made and which engine enforces them. Trino documents file-based access control, Apache Ranger and Open Policy Agent as options, and each has different operating costs. File-based rules are simple to version but harder to scale across many teams. Apache Ranger centralizes policies, including column and row filters, but adds a service to run. Open Policy Agent suits teams that already manage policy as code. Ask the partner how Spark, which has no access control layer of its own, will be governed, how credentials reach object storage, and how policy changes are reviewed, tested and audited. Ask for a sample policy for one sensitive table as a proof of design.

Which authentication method fits our identity provider for the lakehouse?

Trino documents password file, LDAP, OAuth 2.0, Kerberos, certificate and JWT authentication, so the answer should map your identity provider to one of these methods. A strong proposal names the method for interactive users, the method for service accounts and scheduled jobs, and how group membership flows into access rules. It should also cover how revoked staff lose access to the catalog, the query engines and the object storage credentials, since each may be a separate control point. Ask the partner which identities are human, which are machine, and how long-lived secrets are avoided. Vague answers such as "we integrate with your directory" do not show who can query what.

How should lineage and ownership be recorded in a data catalog?

A good answer names a metadata platform, such as DataHub or OpenMetadata (both Apache-2.0), and explains how lineage is captured from each engine and orchestrator. Ask how each dataset gets a named owner, how ownership changes are recorded, and whether lineage comes from query logs, pipeline metadata or manual entries. Column-level lineage is harder to capture than table-level lineage, so ask which level is in scope for the first release and what is deferred. Also ask how the catalog connects to access policies, so an owner can approve or review access requests. Request the partner's plan for keeping metadata current when pipelines change, since stale lineage erodes trust quickly.

Does the catalog design hold for both Iceberg and Delta Lake?

Iceberg and Delta Lake are different open table formats, and a catalog design that works for one may not cover the other without extra components. Iceberg is Apache-2.0 licensed and supports schema evolution without rewriting tables, hidden partitioning and time travel. Trino's Iceberg connector reads Parquet, ORC or Avro and supports format versions 1 and 2, with version 3 listed as experimental. Ask the partner which format is the system of record, whether Delta tables are in scope, and how a second format would be registered and governed. Ask them to name the format version they will use and state how they will handle future format upgrades.

How is the catalog backed up, and how is it restored after a failure?

The catalog holds the pointers to every table, so losing it can make data unreadable even when object storage is intact. A good answer covers the catalog's backing database, its backup schedule, how backups are encrypted and stored in a separate location, and how often a restore is tested. Ask the partner to describe a restore procedure in steps, including how table metadata is reconciled with the files in object storage after recovery. Ask whether catalog backups are included in the fixed scope or in an ongoing support arrangement. Vague answers like "the database is replicated" do not cover accidental deletes or corrupted metadata.

Which catalog and access evidence can we show an auditor?

Ask the partner to list the logs and records that the design produces, such as catalog change history, access policy versions, authentication events and query logs. Each should have a named retention period set by your policy, and the partner should explain where they are stored and who can read them. The EU AI Act requires automatic logging for high-risk AI systems, which may matter if lakehouse data feeds such systems, and Article 19 sets a six-month minimum retention for those logs unless other law applies. Ask the partner which evidence they will prepare for your compliance assessments and state clearly that auditors and certification bodies, not the implementer, issue the findings.

Who owns policy changes after the lakehouse goes live?

Governance design fails when no one owns the rules after handover. A good answer names the roles that approve access requests, change catalog or policy settings, and review access on a schedule, and it shows how those approvals are recorded. Ask the partner whether your team will maintain the policies or whether they will be handled under an ongoing support arrangement, and what that arrangement covers. Also ask how a new data domain is onboarded, including its owner, schema standards and access rules. The answer should show a process your staff can run without the implementer in the loop.

How to test table maintenance and query performance

Table maintenance and query performance decide whether a lakehouse stays fast and affordable after go-live. The cards below set out the tests a partner should run on your own data, the settings that keep long jobs alive, and the evidence each test should leave behind.

Compaction and snapshot expiry schedules

Small files and old snapshots slow queries and inflate storage on Iceberg tables. Ask the partner to state the compaction target file size, the frequency of compaction and snapshot expiry jobs, and the retention window per table. Check how these jobs are scheduled in Airflow or Dagster, how they are monitored, and whether a failed run alerts someone. Ask for the schedule as a written runbook and a before-and-after file count and query latency for a sample table.

Orphan file cleanup

Failed writes and expired snapshots can leave data files that no table metadata references, and these files consume storage without being visible to queries. Ask the partner how orphan detection is run, how often, and what safety window protects files that are still being written. Require a dry run that lists candidate files before any deletion, and ask for a report showing bytes reclaimed per table. Confirm that cleanup runs against the object storage bucket the catalog actually uses.

Time travel retention

Time travel lets analysts and auditors query a table as it was at an earlier snapshot, but only within the retention window you set. Ask the partner to document the retention period for each table class, how it interacts with snapshot expiry and erasure requests, and how rollback is tested. Ask them to demonstrate a query at a historical snapshot in Trino and a rollback on a test table, and to show that the retention setting is enforced by a job rather than by convention.

Benchmark on your own data and query mix

Public benchmarks rarely match your table shapes, join patterns or concurrency. Ask the partner to build the benchmark from a copy of your volumes, a list of your top queries by frequency and cost, and agreed success thresholds written before testing starts. Require the query text, data generator or extract method, engine versions, cluster sizing and raw timings. Ask for a repeatable test harness that your team can rerun after upgrades, not a single slide of results.

Fault-tolerant execution for long batch jobs

Trino's fault-tolerant execution is off by default and must be enabled with a QUERY or TASK retry policy. The TASK policy is the one the documentation recommends for large batch queries, and it requires an exchange manager that spools intermediate data to S3, Azure Blob Storage, Google Cloud Storage or HDFS. Ask the partner which policy they chose and why, which spooling bucket and lifecycle rule they configured, and how a forced worker failure during a long query was tested. Ask for the configuration files and the test log.

Concurrency and workload isolation tests

A lakehouse that answers one query quickly can still stall when dashboards, ingestion and ad hoc analysis run together. Ask the partner to define target concurrency levels, the resource groups or queues that separate workloads, and the point at which latency is considered unacceptable. Require a load test that ramps concurrent users and records queue time, failed queries and coordinator memory. Ask them to confirm which findings lead to sizing changes and which need a change to the design.

How to get a cost model you can compare

Ask every bidder for a cost model that separates one-time work from recurring cost. The one-time column covers discovery, catalog and access design, migration of tables and pipelines, performance testing and handover documentation. The recurring column covers object storage capacity, compute nodes for the query engines (such as Trino or Spark), the engineering time to run the platform, table maintenance such as compaction, snapshot expiry and orphan file cleanup, support contracts and upgrades. A proposal that lists a single total without this split makes it impossible to see what you will still be paying in year two, and it hides the cost of keeping Iceberg or Delta tables healthy after go-live.

Require the model to state its assumptions in writing. Query volume, the number of concurrent users and jobs, peak versus average load, data volume growth, retention periods and the freshness target all drive compute and storage sizing, so each should appear as a named input with the value the bidder used. Ask how the model changes if concurrency doubles or if a dashboard refresh moves from hourly to every few minutes. Confirm which engine sizing the bidder assumed and whether fault-tolerant execution in Trino, which needs an exchange manager spooling to object storage, is included in the compute estimate. Where a figure comes from a vendor or public price sheet, ask for the source and date.

Make the quote follow from the written scope, not the other way round. Ask the partner to tie each cost line to a scope item, and to list what sits outside the quote, such as hardware refresh, licenses for any commercial add-on, or extra support hours beyond the retainer. Ask which party pays for each component, whether that is your cloud account, your data center or the partner's labor. Compare proposals by reading the same assumptions and scope side by side, and treat any line you cannot trace to a scope item or an assumption as a question to resolve before signing.

What a migration and handover plan must contain

The sequence moves analytics from the current warehouse to an open lakehouse without losing reports, and ends with a team and documentation that can run the platform without the partner. Each step produces a record the client can inspect before the next step starts.

Inventory warehouse objects, queries and consumers
The partner lists every schema, table, view, stored procedure, scheduled job, dashboard and downstream extract in the current warehouse, along with the query history that shows who reads what and how often. Your data owners confirm which objects are still in use and which are retired. The step closes with a signed inventory that marks each object as migrate, redesign, retire or keep in place, so scope is fixed before any data moves.
Map tables and transformations to open formats
For each table, the partner records its target format (Apache Iceberg or Delta Lake), partitioning, catalog entry, storage zone and owner, and it translates warehouse-specific SQL and transformation logic into the dialect of the chosen query engine. Data types, null handling, time zones and sort or clustering behavior are documented where they change. The step closes with a mapping register and a list of constructs that could not be translated automatically, each with a decision from your team.
Load historical data and validate row counts
The partner moves historical data into the object storage and open table formats in the agreed order, then compares row counts, checksums or key-level samples between source and target for every table. Load failures, type conversions and rejected records are logged with their resolution. The step closes when each table has a validation report that your data owners have accepted, and when any exceptions are recorded with a documented reason.
Run parallel reports and reconcile results
Existing reports and pipelines run against both the warehouse and the lakehouse for an agreed period, and the partner reconciles outputs metric by metric. Differences are classified as defects, timing differences or accepted changes, and each class has an owner. Your finance, risk or operations users sign off the reports that matter most. The step closes with a reconciliation log showing the result of every compared metric and the sign-off for each business report.
Execute cutover with a tested rollback
The partner switches consumers, scheduled jobs and access grants to the lakehouse in a change window that your change board approves. A rollback procedure is rehearsed before cutover, including how writes are paused and how the warehouse remains readable. Access controls and lineage are checked after the switch. The step closes with a cutover record listing the objects moved, the time of each switch, the checks performed and the rollback test result.
Hand over runbooks, documentation and access
The partner delivers runbooks for backups, table maintenance, catalog operations, upgrades, incident response and access requests, along with architecture and data lineage documentation. Credentials and administrative rights move to your staff, and the partner confirms that every runbook has been executed by someone on your team. The step closes with a handover register that names the owner of each runbook and records which procedures were run by client staff.
Train staff and confirm operating ownership
Platform, data engineering and analytics staff are trained on the components you will run, including query engines, the table catalog, maintenance jobs and the observability tools. The partner observes or supports the first routine operations and then steps back. The step closes with a documented operating model that lists who approves schema changes, who responds to alerts and how support is requested, and with a training attendance and skills record for each named role.

Which engine and platform questions depend on your estate?

A Trino-led lakehouse uses Trino for interactive SQL over tables in Apache Iceberg, with the table catalog and object storage kept separate from the engine. Trino queries data through connectors rather than loading it into its own storage, so the partner must name the object storage and the catalog (an Iceberg REST catalog, Hive Metastore, AWS Glue, Nessie or another) before any other choice is made.

Version policy: Ask how the partner will track Trino releases. Trino release 484 was published on October 15, 2026, after releases 483 and 482, so ask how often upgrades are planned and how the upgrade path is tested against your Iceberg tables.
Java and host requirements: Trino's deployment documentation requires a 64-bit Linux system and a 64-bit Java 25 runtime, minimum 25.0.1. Ask who owns the JDK patch cycle and how it is validated.
Kubernetes or virtual machines: Ask whether the coordinator and workers run on Kubernetes with Helm, or on virtual machines, and whether a dedicated coordinator is planned for your cluster size.
Fault-tolerant execution: Ask whether the TASK retry policy is needed for large batch queries, and if so, which exchange manager spools to S3, Azure Blob Storage, Google Cloud Storage or HDFS.

Confirm the catalog and the authentication and access control model (password file, LDAP, OAuth 2.0, Kerberos, JWT, Apache Ranger or Open Policy Agent) before accepting a Trino plan.

Which partner answers should stop the evaluation?

A lakehouse proposal is only as sound as the answers behind it. The cards below describe partner responses that should pause the evaluation until they are answered in writing, because each one tends to surface later as cost, lock-in or an unusable platform. Check each answer against your own data, your own contracts and the license terms of every component.

Open formats in name only

A partner may say the lakehouse is open because the tables use Apache Iceberg, while the catalog, access control, orchestration or query layer depends on a proprietary service that cannot be moved. Ask which components could be replaced without rewriting tables, which catalog the tables are registered in, and how a full export of data and metadata would run. Apache Iceberg is licensed under the Apache License 2.0, but licensing the format does not make the surrounding stack portable.

Quote before written scope

A price offered before the tables, sources, engines, environments and acceptance tests are defined is a guess. Ask for a written scope that lists the engines, catalog, object storage, identity integration, sources and handover deliverables, and check that the quote follows from that scope. Most fixed-scope engagements define these items before work starts, so a partner who skips this step is asking you to accept unknown risk.

No table maintenance plan

Iceberg tables accumulate small files, old snapshots and orphan files after go-live, and query speed and storage cost degrade without scheduled work. Ask who runs compaction, snapshot expiry and orphan file removal, how often, and how those jobs are monitored. A proposal that covers only the initial load and says nothing about the operating cycle leaves the cost and the performance problem with your team.

Performance claims without your data

Benchmarks run on a vendor dataset say little about your query mix, concurrency or table layout. Ask the partner to run agreed queries on a representative sample of your tables, record latency and resource use, and repeat the test at expected peak concurrency. Trino, Spark and ClickHouse each perform differently across workloads, so the test should decide the engine choice, not the sales deck.

License terms left unchecked

Some components are open source only in part. Airbyte's platform repository is under the Elastic License 2.0, and Metabase's enterprise directory uses the Metabase Commercial License, while the rest of the code is AGPL. A partner who has not reviewed the license of every component, including enterprise directories and extensions, may leave you with a distribution or paid-feature obligation you did not budget for. Ask for a written license inventory before the design is approved.

No named handover owner

A lakehouse that only one consultant understands becomes unmaintainable after the engagement closes. Ask for a named owner on the partner side for handover, a named owner on your side who will run the platform, and a written list of runbooks, configuration, access procedures and upgrade steps. If the partner cannot name the person who will sign off the handover, the handover will not be complete.

How to score partner responses

This table lists the criteria to score each partner response against, with what full marks require and what a failing answer looks like. Set the weights with your own architecture, security and finance teams before the responses arrive, then score each answer against the same evidence. The main takeaway is that a fluent answer without named tables, tests and figures should score lower than a shorter answer that shows them.

Criterion	Weight	Full marks require	A failing answer looks like
Catalog and governance	Set by your team	Named catalog choice (for example Iceberg REST, Hive Metastore or Nessie) with reasons, a stated access control model across engines, and lineage and ownership tooling named by product	Says the catalog is 'to be decided,' offers no access control design, or assumes the warehouse's built-in permissions carry over
Table maintenance	Set by your team	A written schedule for compaction, snapshot expiry, orphan file removal and manifest rewrites, with the owner of each job and how erasure and retention interact with snapshot expiry	Treats maintenance as an afterthought, gives no retention settings, or does not mention old snapshots holding deleted data
Performance evidence	Set by your team	A test plan using your own query mix, data volumes and concurrency, with results shown as measured figures and the engine versions used	Quotes generic speed claims, benchmarks on unrelated data, or refuses to test against your workload before award
Cost model	Set by your team	Separate lines for storage, compute nodes, engineering, support and upgrades, with one-time and recurring items marked and the drivers for each listed	A single lump sum, no sizing assumptions, or a cost comparison to a cloud warehouse that omits engineering and operations
Migration and handover	Set by your team	Phased migration with parallel run and reconciliation criteria, a named handover package (runbooks, configs, access records) and a training plan for your staff	Proposes a one-off cutover, no reconciliation method, or handover described only as 'documentation at the end'
Licensing clarity	Set by your team	Licence for every component (table format, engines, catalog, orchestration, BI) stated with version, plus notes on any enterprise-only features the design depends on	Lists components without licences, relies on a 'community' edition for a required feature without saying so, or cannot name the licence of the BI tool
Support and team	Set by your team	Named roles for the proposed team, documented support hours and escalation, and a plan for keeping engine and table format versions supported	Only names a project manager, gives no support model after go-live, or depends on one unnamed specialist

What do buyers ask before hiring outside help for a lakehouse?

What drives the cost of an open source lakehouse build?

Scope and team effort drive most of the cost, with infrastructure as a separate line. The one-time work covers architecture, catalog and access control design, ingestion from each source, migration of existing tables and validation. The recurring work covers table maintenance, upgrades, monitoring, backups and support. Storage and compute nodes are paid to your cloud or data center provider, while implementation effort is paid to the partner under a written scope. Ask every bidder to split the quote into one-time and recurring lines and to state which items you would own. The cost comparison page explains each component in more detail.

How long does a lakehouse implementation take, and what sets that?

The duration depends on the number of source systems, the volume and freshness of data, the migration load from the current warehouse, and how many compliance zones the design must separate. Access control integration with your identity provider, and the number of query engines you run, also extend the schedule. Ask each partner to show the phases, the milestones where you sign off, and the assumptions behind the timeline. A timeline without named dependencies on your side is a warning sign. Use the requirements checklist to agree these inputs before you compare schedules.

What does the work need from our own team?

Your team supplies decisions, access and data knowledge that no partner can supply for you. Expect to name a data owner for each domain, a security approver for access policies, a cloud or infrastructure owner for storage and networking, and a person who can confirm source system behavior. Business users must agree on the definitions used in reports, since those definitions are the hardest part of a migration. Ask the partner to list these roles in the statement of work so the dependencies are visible before signature.

Can our own team run the lakehouse after handover?

Yes, if the handover includes working runbooks, documented catalog and access configuration, tested backup and restore steps, and training for the engineers who will operate it. Open formats such as Apache Iceberg reduce lock-in because the same tables can be read by Spark, Trino, Flink and other engines. Your team will still own table maintenance, such as compaction, snapshot expiry and orphan file cleanup, plus upgrades of each engine and catalog. Ask the partner what they hand over, what they support after handover, and whether support is a separate retainer.

How does open source licensing affect a lakehouse build?

Most core components are under permissive licenses, such as Apache-2.0 for Apache Iceberg, Trino, Spark and Apache Airflow. Some tools in the wider data stack carry different terms, so each component needs a check. Airbyte's platform repository is under the Elastic License 2.0, and Metabase uses the AGPL outside its enterprise directory with a commercial license for the enterprise code. Ask the partner for a list of every component with its license, the exact directories or features that need a commercial license, and who in your organization approves those terms before the design is fixed.

What does a fixed-scope agreement cover for a lakehouse?

A fixed-scope agreement defines the deliverables, the systems in scope, the acceptance criteria and the timeline before work starts. For a lakehouse, it should name the table format and catalog, the engines, the source systems and migrated tables, the access model, the performance tests, the documentation and the handover sessions. Changes to scope, such as adding sources or new compliance zones, should follow a written change process with a price and timeline. DeployOpen works to written scope and does not publish a price list, so the quote follows from the scope you agree. Ask every partner to show which items are excluded.

How DeployOpen scopes, delivers and supports lakehouse evaluations

DeployOpen works on lakehouse RFP responses as a fixed-scope engagement, with the scope, deliverables and timelines agreed in writing before work starts. We help the buyer turn the questions in this guide into a comparable, evidence-based evaluation of partner proposals.

How we scope

The written scope lists the questions, catalog and governance areas, and workloads each proposal must answer.
We define the reference environment, data volumes and query concurrency that vendors must size against.
The scope states which evidence each proposal must provide, such as test plans, sample configurations and runbooks.

How we build

We build a comparison workbook that scores proposals on catalog design, table maintenance and migration approach.
Where the buyer needs it, we run a performance test on a representative dataset in the buyer's own environment.
We document assumptions behind each cost model line so the buyer can trace how every estimate was produced.

How we hand over

You receive the scored evaluation, the test results and the written assumptions behind each proposal comparison.
Handover includes the requirement traceability matrix, so every question maps to a proposal answer and its evidence.
Open items, risks and questions to confirm with each vendor are recorded for the procurement file.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer once the chosen partner starts work.
We can review the partner's implementation plan against the agreed scope and flag gaps before contract signature.
Follow-up reviews can check that backup, table maintenance and access control evidence stays current after go-live.

Open source data lakehouse RFP questions