Why self-hosted LLM inference is a sovereignty decision, not only a hosting choice
When a data protection officer asks where prompts and outputs go, the answer decides whether an LLM feature can launch at all. Self-hosting open-weight models in EU facilities keeps that data out of third-country transfers, but only if the whole chain is controlled: the serving engine, the gateway, the logs and the people with administrative access. The question is not only which model scores best. Under GDPR Article 44, every transfer of personal data to a third country, including onward transfers, must meet Chapter V conditions, and the EDPB treats remote access from a third country, even just displaying data on a screen for support, as a transfer. A hosted API that stores or processes prompts outside the EEA, or that can be reached by staff abroad, therefore creates exposure before any model output is generated.
Legal exposure also depends on who can compel access. The US CLOUD Act provision at 18 U.S.C. 2713 requires providers to preserve or disclose data in their possession, custody or control regardless of where it is located, and the EDPB has said that where a provider must access data in the clear and a third-country authority's powers exceed what is necessary and proportionate, no effective technical supplementary measure can be envisioned. Your assessment should therefore ask who holds the keys, who operates the cluster, where the prompt and response logs are written and whether any vendor support path reaches them. Confirm these points in writing with each candidate provider rather than assuming a European data center settles them.
For the platform itself, the short answer is vLLM as the serving engine, an OpenAI-compatible gateway in front of it, and Ollama or llama.cpp for small or CPU-bound cases. vLLM is Apache-2.0 licensed, exposes OpenAI-compatible endpoints such as /v1/chat/completions, and has no paid edition, so the recurring cost is hardware, operations and staff time. A gateway such as LiteLLM, which is MIT licensed outside its enterprise directory, can hold virtual keys, budgets and logging in one place, while Ollama (MIT) or llama.cpp suits branch servers, laptops and quantized models where GPUs are not available. Each choice needs a license check, a security review and an agreed upgrade cadence before the first production user.
Which GDPR, Data Act and AI Act requirements shape the inference design
This table maps each EU requirement to what it means for an in-house inference service, what the design must do about it, and what evidence an assessor will ask for. Read it as a list of design questions for the controller's counsel and data protection officer, not as legal conclusions. The main takeaway is that location alone does not settle the question: who can access the data, under which law, and what the logs record all shape the build.
| Requirement | What it means for the inference build | Evidence to prepare |
|---|---|---|
| GDPR Article 44 and Chapter V transfers | Any transfer of personal data to a third country, including onward transfers, needs a Chapter V basis. Fines for infringing Articles 44 to 49 can reach EUR 20 million or 4% of worldwide annual turnover under Article 83(5)(c). | Data flow map showing every location where prompts, outputs and logs are stored or viewed |
| Remote access and cloud storage outside the EEA | EDPB Guidelines 05/2021 treat remote access from a third country, even only displaying data for support, and storage in a cloud outside the EEA as transfers. Vendor support sessions and offshore administrators need to be in scope. | Access list covering vendor and admin accounts, with locations and support procedures |
| EU processor subject to third-country law | Using an EU-based processor is not in itself a transfer, but complying with a third-country authority request would be. Assess that exposure before engaging the provider, since the processor's own legal reach matters. | Documented assessment of the provider's ownership, legal reach and government-request handling |
| Schrems II and the Data Privacy Framework | The Privacy Shield was invalidated in Case C-311/18, while standard contractual clauses remained valid. The EU-US Data Privacy Framework adequacy decision applies only to US organizations on the DPF List, and an appeal in Case C-703/25 P is pending, so its future is uncertain. | Transfer impact assessment and a record of the legal basis relied on, reviewed as the appeal progresses |
| US CLOUD Act reach (18 U.S.C. 2713) | US providers may have to preserve or disclose data in their possession, custody or control whether it sits inside or outside the United States. Where the provider needs data in the clear, EDPB Recommendations 01/2020 say no effective technical supplementary measure may exist if access exceeds what is necessary and proportionate. | Provider's legal analysis of CLOUD Act exposure, plus key-custody design showing who holds decryption keys |
| GDPR Article 48 disclosure orders | A third-country court or authority order to disclose personal data is recognized or enforced only if based on an international agreement in force with the Union or a Member State, such as a mutual legal assistance treaty. Design the request-handling process around that rule. | Written process for receiving, reviewing and refusing or escalating foreign disclosure requests |
| Data Act Article 32 on non-personal data | Providers of data processing services must take measures to prevent third-country governmental access to non-personal data held in the Union where that would conflict with Union or national law (Regulation (EU) 2023/2854). Model weights, configs and operational telemetry can fall in scope. | Provider attestation or contract clauses covering Article 32 measures |
| Data Act switching rights from 12 September 2025 | Customers may switch to another provider or to on-premises ICT infrastructure, with a maximum two-month notice period and a 30-day transition. Switching charges are banned from 12 January 2027 (Articles 25, 29 and 50). Keep models, configs and prompt templates portable, since an OpenAI-compatible endpoint helps. | Exit plan listing exportable artifacts, notice periods in the contract and a tested migration path |
| AI Act logging duties (Articles 12(1) and 19(1)) | High-risk AI systems must technically allow automatic event logging over their lifetime, and providers must keep those logs for at least six months unless other law requires otherwise. Decide early where inference logs are stored, who can read them and whether they contain personal data that GDPR also governs. | Log schema, retention schedule and access controls for gateway and inference logs |
On premises, EU provider or EU region: which hosting option matches which exposure
Hosting in your own data centre gives you the widest control over the stack. You hold the physical access, the encryption keys and the administrative credentials, and no third party sits between your prompts and the GPUs. The GDPR transfer rules in Chapter V only come into play if someone outside the EEA can view or access the data, so the main question becomes who can reach the environment remotely.
- Key custody: Your team generates and stores the disk, database and TLS keys, typically in a hardware security module or an on-premises secret manager.
- Remote administration: Vendor support sessions, offshore administrators and remote monitoring tools count as access. EDPB Guidelines 05/2021 treat even displaying data on a screen for support as a transfer, so restrict vendor access and log it.
- US-parent exposure: Hardware and software vendors with US parents do not hold your data by default, but support contracts and telemetry can give them access paths. Review what each vendor can see.
- Capacity and operations: Your staff carry power, cooling, GPU replacement and patching. These costs recur and rise with utilization and redundancy requirements.
- Contract points: Confirm the support model in writing, including who may connect, from where, and under which logging.
Ask your security team to document every remote access path before you rely on the on-premises label.
An EU colocation facility or a cloud account you own in an EU region places your hardware or virtual machines in a third-party facility while you keep the administrative keys. This model suits teams that need EU-located GPUs without building a data centre. The facility provider handles space, power and physical security, which means its staff have physical proximity to your racks, so the contract must define what they can and cannot touch.
- Key custody: Keep encryption keys and root credentials in your own control, ideally in a key management service you operate or in hardware you hold, not in the provider's default key store.
- Remote hands: Facility staff who reboot or swap hardware are a form of access. Require escorted access and records of each visit.
- US-parent exposure: Check the legal group of your colocation or cloud operator. If its parent is subject to US law, the CLOUD Act (18 U.S.C. 2713) can reach data in its possession, custody or control wherever it is located, so assess that exposure as the EDPB recommends for processors subject to third-country law.
- Article 48 safeguard: A third-country order is enforceable under GDPR Article 48 only where an international agreement such as a mutual legal assistance treaty applies. Ask the provider how it handles such requests and whether it commits to challenge them.
- Contract points: Confirm the data location, sub-processor list, notice of government requests, and whether the provider's support staff can view tenant data.
The main unknown is the legal control of the operator, which you must confirm with the provider rather than assume from its location.
An EU-based provider or the EU region of a larger cloud platform runs the infrastructure for you, and the data stays in the EEA at rest. Location alone does not settle the question, because the provider's ownership, its support model and the legal reach of its parent company determine who can compel access. Hyperscaler EU regions often include sovereignty features, but the feature list and the legal arrangements vary, so read each contract rather than the marketing.
- Key custody: Options range from provider-managed keys to customer-managed keys held in an external store. Customer-held keys reduce, but do not remove, the risk of compelled disclosure, because the provider still processes plaintext during inference.
- Remote administration: Provider engineers in third countries who access the environment create transfers under EDPB Guidelines 05/2021. Ask for a map of where support staff sit and how access is approved.
- US-parent exposure: EU regions of US-headquartered providers remain within the reach of the CLOUD Act. EDPB Recommendations 01/2020 state that where public authority powers go beyond what is necessary and proportionate, no effective technical supplementary measure can be envisioned if the provider needs data in the clear. The EU-US Data Privacy Framework applies only to US organizations on the DPF list, and an appeal in Case C-703/25 P is pending.
- Data Act protections: Article 32 of the Data Act requires providers to take measures against third-country governmental access to non-personal data in the Union. Ask how the provider meets it and what it commits to in writing.
- Contract points: Confirm the transfer mechanism, government access commitments, exit and switching terms under the Data Act, and audit rights.
If a US-parent provider cannot give written commitments that match your exposure, treat its EU region as a transfer risk and document the assessment.
Which open-source serving and gateway platforms fit an EU-controlled setup
An EU-controlled setup is only as sovereign as its weakest component, so each platform below has to keep prompts, outputs and logs on infrastructure you control. The cards cover the serving engines, the gateway and the interfaces that sit in front of them, and the licence terms you must check before you deploy any of them for staff or customers.
vLLM for multi-GPU serving
vLLM is the option to evaluate first when one model must serve many concurrent users on your own GPUs. It is Apache-2.0 licensed, its v0.31.0 release was published on October 5, 2026, and it exposes OpenAI-compatible endpoints so existing applications can point at your servers. Multi-node deployments send traffic between nodes that is insecure by default, so the node network must be isolated and encrypted by your team. Confirm the security controls in the vLLM documentation before exposing any endpoint beyond the internal network.
Ollama for smaller teams
Ollama is MIT licensed and suits teams that want to run a handful of models quickly on a single server or workstation without a dedicated inference engineer. Its v0.40.5 release was published on October 14, 2026. It is a practical fit for pilots and departmental tools, but check whether its endpoint authentication, access logging and concurrency behavior meet your requirements before it serves a larger user base. Plan for a move to vLLM if concurrency or multi-GPU scaling becomes the constraint.
llama.cpp for CPU and edge
llama.cpp runs quantized models on CPUs, which matters when a branch office, plant or laptop cannot host a GPU or when GPUs are not permitted in that environment. Its open-source license lets you keep weights and inference entirely local, but you must confirm the license of each model file separately, since the engine license does not cover the weights. Expect smaller models and slower responses, and plan how you will patch and monitor many small nodes.
LiteLLM as the controlled gateway
LiteLLM sits between applications and model endpoints to handle routing, virtual keys, per-team budgets and logging, so every request passes one controlled point. Its core is MIT licensed, but the enterprise/ directory uses a separate licence, and SSO beyond five users and features such as SCIM and audit logs need an Enterprise license. Virtual keys require PostgreSQL, and Redis is needed once more than one proxy instance runs. Confirm which features you need before you pick the license tier.
Open WebUI and LibreChat as front ends
Open WebUI and LibreChat give staff a chat interface that keeps conversations and uploaded files on your infrastructure. LibreChat is MIT licensed, while Open WebUI is under its own license whose branding clause restricts altering or removing the Open WebUI branding, except for deployments with no more than 50 end users in any rolling 30-day period, with written permission, or under other listed exceptions. Read clause 4 before you rebrand the interface for a wider audience. Confirm that your deployment model fits the exceptions before you roll it out.
Langfuse for prompt records
Langfuse records prompts, outputs, latency and cost in storage you operate, which supports the logging evidence that high-risk AI obligations may require. Its core is MIT licensed, but its ee directories are under separate terms, and ClickHouse, Inc. holds the copyright. Phoenix is an alternative under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service, so a partner running it for your clients needs a separate review. Decide your retention period for prompt records against your legal basis and the AI Act log rules.
How to lock down the endpoints vLLM leaves open
The --api-key option is not a complete control for a vLLM server that holds EU personal data. According to the vLLM security documentation, the key protects endpoints under /v1, /v2, /inference and /cohere, but not /invocations, /pause, /resume, /abort_requests, /metrics or /health. The documentation advises against relying on the key alone and recommends a reverse proxy that forwards only the endpoints meant for users, such as /v1/chat/completions and /v1/embeddings. Treat that allowlist as a sovereignty control: anything not on it should return an error at the proxy. Before go-live, have your security engineers send requests to every route the server exposes from a user network segment and confirm that each unlisted path is refused.
Multi-node deployments add a second exposure. The vLLM security docs state that communication between nodes is insecure by default, with no authorization or encryption, so the Ray or multiprocessing traffic that splits a large model across machines must sit on an isolated network segment that application clients cannot reach. Confirm this with your network team: the inference nodes should accept inter-node traffic only from each other, and the public or internal API should terminate at the proxy. Terminate TLS at that proxy with certificates from your own internal certificate authority, so prompts are encrypted between application servers and the model service, and verify that plain HTTP is disabled on the user-facing port.
Egress blocking is the control most teams skip. An application that is misconfigured with a hosted API key can send prompts out of the building without any error on the inference side, so the model hosts and gateway should have default-deny outbound rules, with only the internal model repository and telemetry collectors you have named as exceptions. Test it by attempting a call to a hosted provider endpoint from each inference node and from the gateway. Metrics deserve the same residency rules as prompts. The /metrics endpoint exposes request counts, token throughput and latency histograms, which can reveal usage by team and time, and logs from the proxy and engine can contain prompt fragments. Decide where Prometheus scrapes and log stores run, and classify these records as personal data where they carry identifiers.
How the rollout moves from transfer assessment to a production EU endpoint
The sequence starts with legal and transfer decisions, then tests the technical design in a non-production zone, and ends with a production endpoint that has documented controls, logs and a patch routine. Each phase closes with a named output that the next phase depends on.
Map transfers with counsel and the DPO
The controller's counsel and data protection officer list every flow that could carry personal data: prompts, uploaded files, outputs, logs, support sessions and backups. They mark which flows leave the EEA, including remote access from a third country, and which vendors or staff could be subject to third-country access demands. The closing output is a signed transfer map with a documented position on each flow, which the rest of the rollout uses as its scope.
Review model licences and build the inventory
The team lists candidate open-weight models and checks each model licence for commercial use, redistribution, acceptable use terms and any restriction on hosting for third parties. The same review covers the serving and interface software, such as vLLM, Ollama, Open WebUI or LibreChat, since their licence terms differ, including branding clauses. The closing output is a model and component inventory with licence, version, source and owner for each item.
Decide hosting and review EU provider contracts
The organization chooses between on-premises hardware, its own EU cloud accounts, a colocation facility or an EU hosting provider. If an EU provider is involved, counsel reviews the processing terms, sub-processor list, data location commitments, support access model and the provider's exposure to third-country legal demands, as the EDPB guidelines recommend assessing before engagement. The closing output is a hosting decision record and a signed contract position for every provider in the data path.
Build the reference stack in a non-production zone
DeployOpen builds the reference environment inside the chosen infrastructure, with the serving engine, the gateway, the interface, the metrics endpoint and the storage, using only the images and weights that passed the inventory review. Multi-node networking is isolated, since vLLM's documentation states node-to-node traffic is unauthenticated and unencrypted by default. The closing output is a documented build with a bill of materials and a tested rollback.
Test egress and endpoint exposure
The security team runs egress tests to confirm the inference nodes cannot reach external model providers, telemetry hosts or package registries at runtime. They also test the endpoint surface: the vLLM API key does not cover every route, so a reverse proxy must allowlist only the routes meant for users and block operational routes such as metrics, health and control endpoints from general access. The closing output is a test report with findings, fixes and a retest result.
Design logging and retention for AI Act Article 19
Where the system could be high-risk under the EU AI Act, the design must allow automatic event logging over its lifetime, and Article 19(1) requires providers to keep those logs for at least six months unless other law provides otherwise. The team defines which events are logged, where logs are stored, how prompts are minimized or redacted, who can read them and how deletion requests interact with retention. The closing output is an approved logging standard and a retention schedule signed by the DPO and the system owner.
Run a pilot with named users
A limited group of named users, each assigned by the business owner, uses the endpoint on approved data classes only. The team reviews access, budgets, latency, error patterns, logs and any prompts that raised data handling questions, and records changes to the transfer map if new data types appear. The closing output is a pilot report with user sign-off, open issues and a go or no-go recommendation for production.
Cut over to production with a patch cadence
The production endpoint is switched on after the pilot findings are closed, with monitoring, alerting, backup of configuration and weights, and a named operations owner. vLLM ships new minor releases about every two weeks, so the organization sets a patch cadence that tests each release in the non-production zone before promotion, with the transfer map and licence inventory updated at each change. The closing output is a production runbook and a recorded first patch cycle.
Evidence a DPO, auditor or regulator will ask for
This table maps each piece of evidence a data protection officer, auditor or regulator is likely to request to the platform component or process that produces it. Read it as a checklist to set up before go-live, since several items, such as data flow diagrams and transfer assessments, are hard to reconstruct later. DeployOpen prepares this evidence for your review, but independent auditors and certification bodies, not DeployOpen, issue any audit reports or certifications.
| Evidence item | Produced by | What the reviewer checks |
|---|---|---|
| Data flow diagram showing no third-country path | Architecture documentation covering clients, gateway, vLLM or Ollama nodes, storage and backups | Every hop of prompts, outputs, logs and embeddings stays on EU or on-premises infrastructure, with no remote administration from outside the EEA |
| Records of processing and transfer assessments | Your records of processing activities, plus a documented transfer or access risk assessment for any EU provider with non-EU parent or support access | Whether each processor's exposure to third-country law was assessed, and whether support access or cloud storage outside the EEA is excluded |
| Network egress rules and test results | Firewall and cloud security group policies, DNS controls, and egress tests run from inside the inference network | The inference servers cannot reach external model hosts or telemetry endpoints, and the tests are repeatable and dated |
| Endpoint allowlist configuration | Reverse proxy configuration that exposes only the user-facing routes, since vLLM's --api-key option does not cover every endpoint | Routes such as /metrics, /health and administrative paths are not reachable from user networks |
| LiteLLM virtual key and budget records | LiteLLM proxy database (PostgreSQL) holding keys, per-user and per-team budgets, rate limits and model restrictions | Each key maps to a named owner, has an approved model list, and has a recorded review or revocation date |
| Audit logs and retention settings | Gateway request logs, vLLM and proxy access logs, and the retention policy in your logging platform | Logs record who sent what category of request and when, and are kept for the period your policy and any applicable AI Act obligations require |
| Model licence register | Inventory of each open-weight model with its licence, version, source, hash and approved use | Licence terms permit your intended internal or customer use, and each deployed model has an approved entry |
| Change and patch records for vLLM and the gateway | Change tickets, release notes reviewed, test results and deployment logs for each engine and gateway upgrade | Each version in production is traceable to an approved change, and security patches were applied within your stated timeframe |
| Access reviews | Periodic review of administrator accounts, SSO group membership and virtual key owners | Only current staff hold administrative access to the inference nodes, gateway and key database |
Questions to put to an implementation partner before signing
Start with the question that decides whether the project meets its purpose: how will the partner prove that no prompt, output, log or file leaves the EU? Ask for a written data flow diagram that names every system that touches prompts, including the GPU hosts, the LiteLLM database, the observability store and any backup target, with the physical location of each. Then ask about remote support access from outside the EU. EDPB Guidelines 05/2021 treat remote access from a third country as a transfer, even when the engineer only views data on a screen. A partner who answers this question vaguely, or who says support is handled "from our global team as needed", has not thought through the transfer analysis, and that is a red flag. Ask who has standing access, through which bastion, from which countries, and how each session is recorded and approved.
Next, ask how they will handle the vLLM endpoints the engine leaves open. The vLLM security documentation says the --api-key option does not protect /invocations, /pause, /resume, /abort_requests, /metrics or /health, and recommends a reverse proxy that allowlists only the endpoints meant for users. Ask to see the proxy configuration, the rule set that blocks the control endpoints, and the test they run after each change. Also ask how multi-node communication is isolated, since the same documentation states that inter-node traffic is insecure by default. For the gateway, ask who holds the LITELLM_MASTER_KEY, where it is stored, how many people can read it, and how it is rotated. Ask how model licences are checked for commercial use before any weights go into production, and which records the partner keeps.
Finally, ask how GPU capacity is monitored, how engine releases are tested before rollout, and what happens if you change partners. Ask which metrics feed capacity alerts, such as the Prometheus metrics vLLM exposes at /metrics, and who is paged. Ask for the staging process that a new vLLM minor release goes through, given that releases arrive about every two weeks. On switching, the Data Act gives customers the right to move to another provider or to on-premises infrastructure, with a maximum two-month notice period and a 30-day transition under Articles 25 and 29. Ask the partner to commit in the contract to exporting configurations, model artifacts, keys and logs in usable formats, and confirm which of those items are excluded.
Questions buyers ask before hiring help for EU self-hosted inference
What drives the cost and effort of an EU self-hosted inference platform?
The biggest drivers are the accelerator capacity you buy or rent, the facility or cloud region that holds it, and the people who build and run the service. Engine software such as vLLM has no paid edition, so the license line is zero, but GPUs, power, storage, networking and on-call engineering are real costs paid to hardware vendors, colocation or cloud providers, and staff or a partner. Effort rises with the number of models, concurrent users, the need for multi-node serving and the depth of audit evidence you must produce. Ask any partner for a quote that splits hardware, hosting, implementation, operations and support, and states which items are fixed and which depend on usage.
Which costs are one-time and which recur for self-hosted LLM inference?
One-time costs usually include architecture, hardware selection, initial installation, network and identity integration, hardening, security testing and documentation. Recurring costs include GPU lease or depreciation, power and facility fees, storage and backups, software upgrades, monitoring, model updates, staff time and any support retainer. Upgrades matter because vLLM ships new minor releases about every two weeks, with v0.31.0 published on October 5, 2026, so you should budget for regular patching rather than a single release. Ask a partner to show which items recur, how often each is reviewed and what changes the monthly figure.
How long does a self-hosted inference rollout take in the EU?
Length depends mainly on hardware lead times, the facility or cloud region you choose, the number of models and user groups, and how much compliance evidence must be ready at go-live. Procurement of GPUs and the data processing agreements or transfer assessments with any provider often take longer than the software installation itself. Identity integration, network segmentation and a documented decision on logging for the EU AI Act, including the six-month log retention under Article 19(1), can also extend the plan. Request a timeline in the written scope with named dependencies, and confirm lead times for accelerators with your hardware vendor.
What does the work require from our own team?
Expect to supply a named business owner, an infrastructure or platform contact, a security contact, and someone who can approve data flows and access groups. Your team also needs to decide which data classes may reach the model, which users get access and how long logs are kept. Legal or the data protection officer should review the transfer analysis and any processor contracts, since DeployOpen prepares evidence but does not give legal advice or issue audit reports. Ask a partner to list these roles, the hours they expect from each, and the decisions that must be made before build work starts.
Can an in-house team run EU self-hosted inference alone?
Some teams can, especially those already running GPU clusters, Kubernetes and a model gateway. The harder parts are securing the endpoints, since the vLLM security docs say the --api-key option does not cover every route and recommend a reverse proxy that allowlists only user endpoints, and keeping node-to-node traffic isolated in multi-node deployments because it is insecure by default. Operating the service also means tracking engine releases, model updates, capacity and incident response. Be honest about on-call coverage and skills before deciding. A fixed-scope build with a handover, or a retainer for upgrades and testing, is a common middle path.
Does an EU hosting provider remove the GDPR transfer question?
Not automatically. Under GDPR Article 44, the question is where personal data is accessed and stored, and who can compel access, not only where the company is headquartered. The EDPB Guidelines 05/2021 treat remote access from a third country, even just displaying data on a screen for support, as a transfer, and storage in a cloud outside the EEA as a transfer as well. An EU provider owned by a non-EU parent may still face third-country legal demands, such as under the US CLOUD Act at 18 U.S.C. 2713. Ask the provider for its access controls, support locations, and how it responds to foreign government requests, then take that to counsel.
Does self-hosting remove our GDPR obligations?
No. Running models on your own infrastructure removes some transfer exposure, but GDPR still applies to any personal data you process, including prompts, outputs, logs and files. You still need a lawful basis, data minimization, retention rules, access controls, security of processing, records and responses to data subject requests. Fines for infringements of Articles 44 to 49 can reach EUR 20 million or 4% of worldwide annual turnover under Article 83(5)(c), so the duties remain your responsibility. Ask a partner which GDPR obligations its scope covers and which stay with your legal and privacy team.
Can DeployOpen guarantee EU data sovereignty for our inference platform?
DeployOpen plans, implements, hardens and documents self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks, and it prepares systems and evidence for compliance assessments. It does not issue audit reports, attestations or certifications, and it cannot decide your legal position on transfers. Sovereignty depends on where hardware sits, who holds administrative access, which providers you use and how your contracts read. Ask for a written scope that names the controls DeployOpen will configure, the evidence it will hand over, and the decisions that remain with you and your counsel.
How DeployOpen scopes and supports EU-hosted inference
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure, including EU data centers and EU cloud accounts. For EU-hosted inference, that means agreeing in writing where prompts, outputs and logs are stored, who can reach the endpoints, and which evidence the organization will need before staff rely on the service.
How we scope
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts
- Scope records the EU locations for inference hosts, model storage, logs and backups, and who can access each
- Scope lists the models, licences and engine versions to be deployed, checked against the organization's use case
How we build
- Deploy the inference engine, such as vLLM or Ollama, on GPUs or servers the client controls in the EU
- Place an authenticated reverse proxy in front of the engine so only user-facing endpoints are reachable
- Isolate multi-node traffic on a private network, since the engine's inter-node communication is insecure by default
How we hand over
- Deliver architecture, network diagrams and data flow documentation showing where personal data is processed
- Provide runbooks for engine upgrades, model replacement, capacity monitoring and restoring model artifacts
- Document access control, logging and retention settings so the client's assessors can review them directly
How we support
- Ongoing support, upgrades and continuous security testing are available as retainers after handover
- Retainers can cover tracking of engine releases, which vLLM publishes about every two weeks
- Security testing can include the inference endpoints, gateway and access controls on a scheduled basis
