Open source data lakehouse for EU data sovereignty

An EU-controlled lakehouse removes most of the cloud warehouse exposure, but the transfers that remain, from remote support to foreign-law access to your provider, must be identified and documented. This page explains how to build and evidence that.

Book a callTalk to us
On this page

Why an open source data lakehouse for EU data sovereignty starts with the transfers you still have

Analytics that sits in a US-hosted cloud warehouse raises one question before any technical choice: can the data stay under EU control? Open table formats such as Apache Iceberg on EU-controlled object storage, queried by a self-hosted engine such as Trino and governed by a self-hosted catalog, can remove most of the transfer exposure. They do not remove all of it. The remaining transfers (remote administrative access, vendor support sessions, and reach of foreign law over the provider) must be mapped, justified and documented before anyone calls the design sovereign. Nothing in this stack makes an organization compliant on its own.

This scenario differs from a general lakehouse build because the transfer analysis drives the architecture, not the other way round. A lakehouse built for performance can run analytics on a managed service in any region. A sovereignty-driven build asks where each layer runs and who can reach it: the object storage bucket, the catalog database, the query coordinator and workers, the orchestration and BI tools, and the people with administrative rights to all of them. The table format matters less than these locations. Iceberg and Delta both keep data in open files, but a managed service that holds the keys, the catalog or the logs can still be a transfer point. Trino reads Iceberg tables through its Iceberg connector and needs separate object storage and a table catalog, so the team has to choose and host that catalog explicitly. Ask any proposed partner to show, layer by layer, the country of storage, the country of operator access and the jurisdiction of each provider.

What an open source data lakehouse for EU data sovereignty must satisfy

This table maps each legal and contractual requirement that shapes an EU-controlled lakehouse to the design decision it affects. Read each row as a question your architecture must answer: where data sits, who can reach it, and what your providers can be compelled to do. Legal interpretation belongs to counsel, so confirm each reading with them before the design is frozen.

RequirementSourceWhat it means for the build
Every transfer to a third country, including onward transfers, must meet Chapter V conditionsGDPR Article 44Map every system that copies or exposes lakehouse data outside the EEA, including backups, logs and vendor support tooling, and record each as a transfer or a non-transfer.
Fines of up to EUR 20 million or 4% of worldwide annual turnover for Articles 44 to 49 infringementsGDPR Article 83(5)(c)Treat undocumented transfers as a board-level risk, so the data flow register and transfer decisions need named owners and sign-off.
Remote access from a third country, even only viewing data on a screen, counts as a transferEDPB Guidelines 05/2021 (version 2.0)Restrict administrator and vendor support sessions to EEA-based staff or controlled access paths, and log every privileged session.
Storage in a cloud outside the EEA counts as a transferEDPB Guidelines 05/2021 (version 2.0)Pin object storage, catalog databases and backup targets to EEA regions and verify replication settings do not cross the boundary.
Privacy Shield was invalidated, while standard contractual clauses remained validCJEU Case C-311/18 (Schrems II), press release 91/20Do not rely on a transfer mechanism that has been struck down. Use standard contractual clauses only with a documented transfer impact assessment.
The Data Privacy Framework adequacy decision applies only to US organizations on the DPF List, and its appeal is pendingCommission Implementing Decision (EU) 2023/1795; Case C-703/25 PCheck each provider's DPF listing status, but do not make the design depend on it while the appeal is pending. Plan a fallback that needs no adequacy decision.
No effective technical supplementary measure exists where a provider needs data in the clear and foreign powers exceed what is necessary and proportionateEDPB Recommendations 01/2020 (version 2.0)If a provider or support team must see plaintext data, a technical control such as encryption with keys held in the EEA may not be enough. Assess the exposure before engaging the provider.
US CLOUD Act reaches data in a provider's possession, custody or control regardless of location18 U.S.C. 2713Ask providers whether a US parent or affiliate can reach EEA-hosted data. Keep encryption keys and catalog administration under EEA control where possible.
Third-country disclosure orders are recognized only under an international agreementGDPR Article 48Write contract clauses that require the provider to challenge foreign requests and notify you, where the law allows, and to route requests through an agreed legal basis.
Providers must prevent third-country governmental access to non-personal data held in the UnionEU Data Act Article 32 (Regulation (EU) 2023/2854)Extend the same access controls to non-personal datasets, such as aggregated operational or manufacturing data stored in the lakehouse.
Switching rights apply from 12 September 2025, with switching charges banned from 12 January 2027EU Data Act Articles 25, 29 and 50Keep tables in open formats such as Iceberg or Delta and document export paths, so you can move to another provider or on-premises infrastructure without proprietary exit fees.

Which open source components keep the lakehouse portable and under EU control

A sovereignty-driven lakehouse depends on choosing components that can run entirely in EU infrastructure and that the organization can move later. The cards below cover the table format, the query engine, orchestration, catalog and governance, and the analytics layer, with the license and version facts that matter for each.

Apache Iceberg as the table format

Iceberg is the open table layer that keeps data portable across engines. Its Apache License 2.0 allows self-hosting without a vendor relationship, and the documentation menu lists 1.12.0 as the latest release. Schema evolution without table rewrites, hidden partitioning, time travel and rollback make it workable for regulated reporting. Confirm which language implementations (Java reference, PyIceberg, Go, Rust or C++) each writer and reader in your estate needs, and test that every engine reads the same snapshots.

Trino as the self-hosted query engine

Trino is Apache-2.0 licensed and queries data where it sits through connectors, so the storage stays in your EU buckets rather than in a vendor's engine. Its Iceberg connector works with Hive Metastore, AWS Glue, JDBC, REST, Nessie and Snowflake catalogs, selected through iceberg.catalog.type. Fault-tolerant execution with a TASK retry policy needs an exchange manager spooling to S3, Azure Blob, GCS or HDFS, so confirm which of those stores sits inside the EU boundary. Check the Java 25 runtime requirement against your platform baseline.

Airflow or Dagster for orchestration

Pipeline orchestration decides where data is moved and when, so it must run in EU infrastructure with logs and credentials under your control. Apache Airflow (Apache-2.0, 3.3.2) and Dagster (Apache-2.0, 1.13.26) are both open source and can be deployed in your own cloud accounts or data centers. Choose one based on your team's skills and pipeline style. Confirm that scheduler metadata, task logs and secrets stay in EU storage and that no hosted control plane sits outside the boundary.

Airbyte licensing check before ingestion

Airbyte is a common choice for loading SaaS and database sources into lakehouse tables, but its platform repository is licensed under the Elastic License 2.0. That license restricts providing the software to third parties as a managed service, which may or may not matter for an internal deployment. Have counsel confirm the licensing fit for your intended use before you build on it. Also confirm which connectors are community-maintained and whether each one moves personal data outside the EEA.

DataHub or OpenMetadata for lineage

Sovereignty audits ask where personal data came from, where it flows and who owns each table. DataHub (v1.7.0.1) and OpenMetadata (2.0.5) are both Apache-2.0 catalogs that can record lineage, ownership and classifications for tables in your Iceberg estate. Pick one based on connector coverage for your sources and engines. Confirm that the catalog runs in EU infrastructure and that its search index and metadata store are not sent to an external service.

Superset or Metabase for analytics

The analytics layer is where users see personal data, so access control and hosting location matter as much as features. Apache Superset (Apache-2.0, 6.1.0) is permissively licensed and self-hosted. Metabase is AGPL outside its enterprise directory, which uses the Metabase Commercial License, so confirm which features you need sit in the open source edition. Verify row-level security against Trino's access control options, including file-based rules, Apache Ranger or Open Policy Agent, before exposing sensitive tables.

Catalog choices that decide who can reach the table metadata

A self-hosted Iceberg REST catalog runs as a service in your own infrastructure and exposes the Iceberg REST interface to engines. In Trino, the Iceberg connector is set with iceberg.catalog.type to the REST value, and the connector then reads table metadata through that endpoint. The metadata service, its database and its network path all stay under your control, which makes this the option where the sovereignty question is easiest to answer on paper, provided the operators and hosting location are inside the boundary you have defined.

  • Hosting: the service, its metadata database and backups run in your EU accounts or data centers.
  • Read path: Trino, Spark and other engines authenticate to the endpoint, so access is decided by your identity setup.
  • Operators: administrators of the catalog can see every table definition and file path, so their access needs the same controls as the data.
  • Dependencies: confirm the open-source catalog implementation you choose, its database and its support model before committing.
  • Object storage: the catalog points to files on your own EU object storage, which must be separate from the catalog service.

Confirm with your team who holds administrative rights to the catalog and whether its logs record table-level reads.

How to document the transfers that remain after the move

The goal of this section is an inventory in which every transfer that remains after the move is listed with its destination, its safeguard and a named owner. That inventory covers more than storage. It also covers each support path, each sub-processor and each foreign-law exposure that could compel access to the data. A lakehouse on EU-controlled object storage can still generate transfers, and the most common one is remote access from a third country. Under the EDPB Guidelines 05/2021, even displaying data on a screen for support or administration counts as a transfer. Vendor engineers in a non-EEA location who open a session on a production catalog or a Trino coordinator therefore belong in the register, even if no file leaves the EU.

The EDPB guidelines also address a point that often confuses procurement teams. Using an EU processor that is subject to third-country law with extraterritorial effect is not in itself a transfer (Example 12). Complying with a third-country authority's request for data would be one, so the controller should assess that exposure before engaging such a processor. For a lakehouse, the practical questions are which entity holds administrative credentials for the storage account and catalog, which legal process could reach that entity, and whether its contract commits it to challenge overbroad requests. Confirm these answers in writing with the processor and with your counsel. DeployOpen prepares the technical records for this assessment but does not provide the legal analysis itself.

A transfer register entry should record the data categories, the destination country or entity, the access path (for example, interactive query, administrative session, ticket-based support or backup replication), the safeguard relied on, and a review date. Where the safeguard is standard contractual clauses, record the module used under Commission Implementing Decision (EU) 2021/914 and where the signed clauses are filed. Where the recipient is a US organization, record whether reliance rests on the Data Privacy Framework. That reliance is fragile: the adequacy decision in Commission Implementing Decision (EU) 2023/1795 applies only to organizations on the Data Privacy Framework List, and an appeal against the General Court's September 2025 judgment is pending as Case C-703/25 P. Set the review date so that a ruling on that appeal triggers a re-check of every entry that depends on it.

Rollout for an open source data lakehouse for EU data sovereignty

The sequence moves analytics off US-hosted warehouses in a controlled order, so each copy, admin path and support path is mapped before data moves, and the exit test proves the tables can be read outside the original engine.

  1. Map data, copies and access paths

    DeployOpen works with the client's data owners, DPO and counsel to inventory every source system, warehouse copy, backup, export, admin path and vendor support path. Each one is checked against the transfer rules, including storage outside the EEA and remote access from a third country, which the EDPB guidelines treat as transfers. The output is a transfer inventory that marks each flow as stays in the EU, needs a documented transfer basis, or must be removed. The step is done when counsel and the DPO have signed off the inventory.

  2. Design EU-only storage, keys and identity

    The team designs object storage, regions, encryption key custody and catalog placement so that data, metadata and logs stay in EU-controlled infrastructure. It also confirms where the catalog and identity provider run, and who holds administrative access, since US-controlled cloud identity or support staff can create transfers or third-country access exposure. The output is a design record covering regions, key ownership, catalog hosting and privileged access. Counsel should confirm whether any third-country processor still needs an assessment before it is engaged.

  3. Build the self-hosted lakehouse platform

    DeployOpen deploys self-hosted Trino, an Iceberg catalog such as an Iceberg REST catalog or Hive Metastore, and the orchestration layer in the client's infrastructure. Before installation, the team confirms the Java runtime that the chosen Trino release requires, since current Trino documentation specifies a 64-bit Java 25 runtime with a minimum patch level, and checks the cluster layout and fault-tolerance settings. The output is a tested platform with documented configuration, backups and upgrade procedure, verified in a non-production environment first.

  4. Migrate tables and reconcile results

    Tables are migrated from the current US-hosted warehouse into Iceberg format on EU storage. Each table is checked for schema, row counts, null handling, and aggregate totals against the source, and time-travel snapshots are checked for the reporting periods the business needs. Discrepancies are logged and resolved before a table is marked complete. This is where portability is actually tested: the migration reveals which features depend on warehouse-specific behavior. The output is a reconciliation report per table, signed off by the data owner.

  5. Configure access control, lineage and retention

    The team configures authentication, role-based access, and row or column restrictions in the query layer, using the access control options the chosen Trino release supports, such as file-based rules, Apache Ranger or Open Policy Agent. Metadata and lineage are registered in the catalog, and retention schedules are set per dataset and per log type. Access logs are kept according to the retention the DPO specifies. The output is a configuration baseline and a test run showing that unauthorized roles are denied and authorized roles see the expected data.

  6. Assemble the transfer register and evidence pack

    DeployOpen compiles the transfer register from the inventory, listing each remaining third-country flow, its legal basis, any supplementary measures, and the owner who reviews it. The evidence pack contains architecture diagrams, configuration exports, access reviews, backup and restore records, and the reconciliation reports, organized for the client's counsel and DPO. DeployOpen prepares this material, but the legal assessment and conclusions belong to the client's counsel and DPO, and independent auditors make any formal assurance decisions. The output is a handover package that counsel and the DPO accept in writing.

  7. Cut over and prove portability

    Reporting and dashboards are switched to the lakehouse in a planned cutover, with the previous warehouse kept read-only until reconciliation holds. The US-hosted copies are then decommissioned under a documented deletion procedure, covering backups and replicas as well. The exit test reads the same Iceberg tables from a second engine, such as Spark or another Iceberg-capable reader, and compares results with the primary engine. A passing test shows that the data is open and not tied to one vendor's warehouse. The output is a decommissioning record and an exit test report.

Evidence that shows data stayed in the EU and the transfers are documented

This table lists the artifacts an assessor, counsel or supervisory authority is likely to ask for, the question each one answers and the component that produces it. Read it as a checklist to agree before go-live, not as a list of proof you already hold. DeployOpen prepares these records, but independent auditors and legal counsel decide whether they are sufficient.

EvidenceQuestion it answersComponent that produces it
Storage bucket locations and replication settingsAre the Iceberg data files physically in EU regions, and do replicas or backups stay there too?Object storage configuration, bucket policies and replication rules exported from the storage platform
Catalog hosting record and access logsWhere does the Iceberg REST catalog or Hive Metastore run, and who has read or write access to table metadata?Catalog deployment inventory and catalog service audit logs
Trino authentication and access control configurationWhich identities can query which tables, and is access tied to controlled directories or policy engines?Trino configuration for password file, LDAP, OAuth 2.0, Kerberos, certificates or JWT, plus file-based rules, Apache Ranger or Open Policy Agent policies
Remote access logs and support session recordsDid any person outside the EEA view, administer or troubleshoot the platform, even without copying data?Bastion or VPN logs, administrator session recordings and vendor support tickets with access scope
Transfer register with safeguards and review datesWhich remaining transfers exist, on what Chapter V basis, and when were they last reviewed?Controller-maintained register that lists each processor, sub-processor and third-country access path
Decommissioning record for US-hosted copiesWere the copies in the previous warehouse deleted, and can the team show when and how?Change tickets, deletion logs and vendor confirmation for warehouse tables, backups and exports
Sample of Iceberg snapshot metadataWhich data files does each snapshot reference, and do all of them sit in approved locations?Iceberg metadata files (manifest lists and manifests) sampled and checked against the bucket inventory
Exit test result with a second engineCan another engine read the same tables without the original query engine, which supports portability claims?Test report from a second engine such as Spark reading the same Iceberg tables, recorded with versions and results

What to ask an implementation partner before the sovereignty build

Start with the transfer register. Ask the partner to show a written inventory of every flow that could be a transfer under GDPR Chapter V, including remote access from a third country, support tickets that expose data on a screen, backups, logs and any cloud region outside the EEA. Under EDPB Guidelines 05/2021, displaying data remotely from a third country counts as a transfer, so a partner who cannot produce that inventory is not ready for a sovereignty build. Ask who owns the register after handover, who signs off changes to it, and how often it is reviewed. Ask as well how they assess any EU processor that is subject to third-country law, since the same guidelines say complying with a third-country authority request would itself be a transfer.

Next, pin down where the catalog and administrative access run. For an Iceberg or Delta lakehouse, the REST or Hive Metastore catalog, the object storage credentials, the Trino coordinator and the identity provider decide who can read the tables, whatever the storage location. Ask which region and jurisdiction each component runs in, who holds the encryption keys and the root or admin credentials, and whether the vendor or any outside party can reach those planes. Ask for the procedure used when a vendor support engineer needs access, including who approves it, how it is logged and whether it is time-limited. A catalog hosted outside the EU without written disclosure is a red flag. So is a partner who says the design is "automatically compliant" with the GDPR, the Data Act or the EU-US Data Privacy Framework. Ask for the exact clauses and transfer tools they rely on, and for their position on the pending appeal in Case C-703/25 P before accepting any plan that depends on the Data Privacy Framework.

Test exit before you need it. Ask the partner to demonstrate that a second engine can read the same Iceberg tables and their snapshots, for example Trino reading a table written by Spark, and to show the metadata files, catalog entries and data files that would move in a switch. Ask how they will document the switching path under the Data Act, which gives customers the right to move to another provider or to on-premises infrastructure, with a maximum two-month notice period and a 30-day transition, and bans switching charges from 12 January 2027. Finally, ask what they will not certify. A reputable partner prepares controls and evidence for an independent auditor and does not issue attestations or certifications, and it should say so in the statement of work. Require a fixed scope, deliverables and acceptance criteria in writing before any work starts.

What buyers ask about cost, effort and time for an EU-sovereign lakehouse

How much effort does a sovereignty migration from a US-hosted warehouse take?

Effort scales with the inventory, not with a single headline number. The main drivers are how many source systems feed the warehouse, how many copies of each dataset exist, how many transfer paths reach outside the EEA (including remote support access and vendor-hosted backups), and how many tables and downstream dashboards, jobs and data shares depend on them. Each table that moves needs its schema mapped, its load pipeline rebuilt, its access rules re-expressed and its consumers re-pointed and tested. Ask any partner to show the inventory method first and to price the migration from that inventory in the written scope, not from a per-platform estimate.

What costs are one-time and what costs recur for a self-hosted lakehouse?

One-time costs cover the sovereignty assessment, the migration of sources and tables, pipeline and access-control rebuilds, testing and handover documentation. Recurring costs cover EU-controlled object storage, compute for query engines and pipelines, catalog operation, security patching and version upgrades, backups, monitoring, and upkeep of the transfer register as vendors and sub-processors change. Storage and compute are usually billed by the infrastructure provider, while engineering and support are paid as services. In a quote, ask for each line separately and for the assumption behind each recurring item, such as data volume growth and query concurrency.

Can our internal team run Trino, the Iceberg catalog and access control?

Yes, if the team has the skills for each layer, and most organizations need to build some of them first. Operating Trino requires Linux administration, Java runtime management for the supported Java version, cluster sizing, and Kubernetes or VM deployment skills. The Iceberg catalog needs a deliberate choice between options such as a REST catalog, Hive Metastore, AWS Glue, Nessie or the Snowflake catalog, each with its own backup and high-availability needs. Access control requires familiarity with Trino's authentication methods and with file-based, Apache Ranger or Open Policy Agent authorization. Ask a partner which of these skills it will transfer and how.

How long does an EU data sovereignty lakehouse project take?

The duration depends on the inventory results and the migration scope, so no general timeline applies. A small estate with few sources and clear transfer paths will take far less time than one with dozens of sources, undocumented remote access and many dependent reports. The inventory, the legal review of transfer paths, the catalog design and the pilot migration of one domain typically come before a full migration plan is fixed. DeployOpen agrees the phases, deliverables and timeline in writing before work starts, based on what the inventory shows. Ask for the phase gates and the criteria that move work from one phase to the next.

What do we need to provide for the engagement to succeed?

Several internal roles must commit time because the work cannot be done by engineers alone. Legal counsel must confirm which transfer tools apply to each flow and which third-country laws reach each vendor. The data protection officer must own the transfer register and the assessment of exposure to third-country authority requests. Data owners must confirm which tables are in scope, their retention rules and who may access them. Cloud, network and identity teams must grant access to the target environment, the source systems and the monitoring stack. Agree these responsibilities and named contacts in the written scope before the work starts.

Do the EU Data Act switching rights change our cloud warehouse contract terms?

The Data Act gives customers of data processing services the right to switch to another provider or to on-premises ICT infrastructure, and it applies from 12 September 2025 (Regulation (EU) 2023/2854). The Act sets a maximum two-month notice period and a 30-day transition, and it bans switching charges from 12 January 2027 (Articles 25, 29 and 50). These rights affect what you can negotiate, but they do not rewrite your contract automatically. Have counsel review the current agreement for notice periods, exit assistance, data export formats and any remaining switching fees before you plan the cutover date.

How do DeployOpen fixed-scope engagements and retainers fit into the budget?

DeployOpen works mainly through fixed-scope engagements, where scope, deliverables and timelines are agreed in writing before work starts. A quote follows from that written scope, and DeployOpen does not publish a price list. Typical scope items for this topic include the sovereignty inventory, the transfer register design, the catalog and access-control build, the pilot migration and the handover. After go-live, ongoing support, upgrades and continuous testing are available as retainers, which are the usual vehicle for the recurring operations work. Ask for the scope and the retainer terms as separate line items so the one-time and recurring costs stay visible.

How DeployOpen scopes, builds and supports an EU-sovereign lakehouse

DeployOpen plans, implements, integrates, hardens, documents and supports self-hosted open-source platforms inside the client's own cloud accounts, data centers or air-gapped networks. For an EU data sovereignty lakehouse, that means agreeing which data stays in EU-controlled storage, which catalog and engines run there, and which transfers remain to be documented before any table is built.

How we scope

  • Inventory every analytics workload, source system and place where personal data is stored or accessed today.
  • Map which datasets can move to EU-controlled object storage and which remote access paths would count as transfers.
  • Agree the table format, catalog, query engines and access model in a written scope before work starts.
  • Confirm with the client's legal and privacy team which third-country exposures they will accept and document.

How we build

  • Deploy object storage, the chosen catalog and Trino or Spark inside the client's own infrastructure.
  • Configure Iceberg or Delta tables so schema changes, time travel and rollback are tested against the agreed design.
  • Set up authentication, file-based or policy-based access control and audit logging for every query engine.
  • Test table maintenance, snapshot expiry and fault-tolerant execution settings against the expected workloads.

How we hand over

  • Deliver written architecture, data flow and transfer register documents covering every remaining third-country path.
  • Provide runbooks for catalog operations, snapshot retention, backups and restore procedures.
  • Train the client's platform and data teams to operate upgrades and access reviews without DeployOpen in the loop.
  • Record evidence of controls in a form that independent auditors or assessors can review.

How we support

  • Ongoing support, upgrades and continuous testing are available as a retainer after handover.
  • Review each new engine release, connector change or catalog upgrade before it reaches production.
  • Repeat transfer assessments when new vendors, regions or support access paths are introduced.
  • Confirm response times, upgrade windows and coverage scope in the retainer agreement before signing.

Related

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.
Book a callhello@deployopen.com