Self-hosted OpenAI API alternative for EU data residency

Self-hosted OpenAI API alternative for EU data residency: when it is the right answer

For an EU team facing a residency or legal review, the short answer is this: open-weight models served on vLLM in your own EU datacenter or cloud region, behind a self-hosted gateway such as LiteLLM, keep prompt and output processing in the EU under your control. OpenAI's European residency option is a different arrangement. It requires sales approval and abuse monitoring controls for regional processing, so the question your reviewers should ask is not only where the data is stored but who can access it, under which contract, and whether a third party's monitoring or support staff can view it.

Start by mapping the access paths for each option. Under a hosted API, the processor's own systems and personnel sit in the chain, and a residency commitment does not by itself settle questions about remote access from outside the EEA or about compelled disclosure under foreign law. The EDPB's Guidelines 05/2021 treat remote access from a third country, even only displaying data on a screen for support, as a transfer. Ask the vendor in writing which personnel can reach regional data, which abuse monitoring applies to your account, and whether any step runs outside the EU. In a self-hosted setup, the answer sits inside your own access model: who holds cluster credentials, where logs are kept and which support contracts give vendors any access at all.

Self-hosting does not remove legal work. Your GDPR processor analysis still covers the cloud provider hosting the GPUs, any monitoring SaaS that receives prompts, and any gateway vendor, and the EDPB's Recommendations 01/2020 warn that a technical measure may not help if a foreign authority's powers exceed what is necessary. Confirm with your counsel and your cloud provider where data is stored, who has administrative access, and how compelled-access requests are handled. The Data Act's switching rights, applicable since 12 September 2025, are also worth noting in your exit planning.

What GDPR and EU rules require for LLM traffic

Five rules decide where prompts, outputs and logs can lawfully go when an EU team calls an LLM. Some apply to personal data under GDPR, others to non-personal data under the Data Act, and together they determine whether a regional hosted endpoint or a self-hosted model in your own EU region fits your obligations.

Article 44 transfer conditions

Article 44 requires every transfer of personal data to a third country, including onward transfers, to meet the conditions in Chapter V. Prompts, outputs and trace logs that contain personal data count. Controllers should map every place a request is processed, cached, logged or support-accessed, then document which Chapter V mechanism covers each hop. Infringements of Articles 44 to 49 can draw fines of up to EUR 20 million or 4% of worldwide annual turnover under Article 83(5)(c).

Remote access counts as transfer

EDPB Guidelines 05/2021 state that remote access from a third country is a transfer, even when staff only view data on a screen for support or administration. Storage in a cloud outside the EEA is also a transfer. A regional EU endpoint does not settle the question if vendor engineers in another country can open logs or sessions. Ask the provider where support staff sit, what access paths exist and whether those paths are logged and restricted.

Schrems II and the Data Privacy Framework

The Court of Justice invalidated the EU-US Privacy Shield in Case C-311/18 (Schrems II) while keeping standard contractual clauses valid. Commission Implementing Decision (EU) 2023/1795 finds adequacy only for US organizations on the Data Privacy Framework List, and an appeal against the General Court's September 2025 judgment is pending as Case C-703/25 P. Check whether your provider is certified, what transfer tool backs non-certified flows, and whether your legal team has a fallback if the adequacy finding is overturned.

Article 48 and third-country access requests

Article 48 allows a foreign court order or administrative demand for disclosure to be recognized or enforced only when it rests on an international agreement, such as a mutual legal assistance treaty. Separately, the US CLOUD Act at 18 U.S.C. 2713 requires US-based providers to preserve or disclose data in their possession, custody or control, wherever it is stored. EDPB Recommendations 01/2020 state that where a provider must access data in the clear and the destination's powers go beyond what is necessary and proportionate, no effective technical supplementary measure is envisioned. Ask providers how they respond to government requests and whether their keys and logs sit under your control.

Data Act Article 32 and switching rights

Article 32 of the Data Act requires providers of data processing services to take measures that prevent third-country governmental access to non-personal data held in the Union where that access would conflict with EU or national law. The Data Act has applied since 12 September 2025 and gives customers the right to switch to another provider or to on-premises infrastructure, with a maximum two-month notice period and a 30-day transition. Switching charges are banned from 12 January 2027. Write these rights into your contract so an exit to self-hosted infrastructure stays practical.

What OpenAI's European residency keeps in the EU and what it does not

OpenAI's European residency option is not a switch you can assume is on. Regional processing in Europe needs sales approval, and it comes with abuse monitoring controls that govern how requests are reviewed. Before you rely on it, get the scope in writing from OpenAI: which data classes are processed regionally, which endpoints and features are covered, what retention applies to prompts, completions, files and logs, and whether abuse monitoring reviews content outside the EU. Ask for the same answers for any feature your applications use, because coverage is not guaranteed to be uniform across the API.

The questions that matter most for GDPR are the ones the processing terms do not answer on their own. Ask who at OpenAI can access data in the clear, including support staff and engineers, and from which countries that access happens. Under the EDPB Guidelines 05/2021, remote access from a third country, even only displaying data on a screen for support, is a transfer. Also ask about onward processing by sub-processors and any cloud hosting outside the EEA. Confirm how any third-country authority request would be handled. The EDPB Recommendations 01/2020 say that where a provider needs data in the clear and foreign public authority powers go beyond what is necessary and proportionate, no effective technical supplementary measure can be envisioned, so this exposure needs a documented assessment.

Self-hosting moves these questions to your own infrastructure, but it does not remove them. A vLLM or Ollama server running in your EU region still needs its logs, traces and prompt stores reviewed, and any managed cloud service you use underneath it brings its own transfer analysis. Record your findings in the records of processing and the transfer impact assessment, and have counsel or your data protection officer confirm the conclusions. The contract terms that govern OpenAI's regional processing should be reviewed by your legal team, not inferred from this page.

Which open-source options keep processing in the EU

vLLM is the component that actually holds prompts and generates completions, so it determines where personal data is processed. Licensed under Apache-2.0, it has no paid edition, which means the residency question depends on where you deploy it, not on a vendor's regional setting. Run it on GPU capacity in an EU datacenter or EU cloud region that you contract for, and prompts and outputs stay there unless your own configuration sends them elsewhere. Its vllm serve server exposes OpenAI-compatible endpoints such as /v1/chat/completions and /v1/embeddings, so applications built for the OpenAI API can point at an EU endpoint. Prometheus-format metrics at /metrics cover request counts, token throughput, latency and KV cache use, and the Kubernetes docs list native manifests, Helm, the production-stack, KServe and KubeRay among deployment options.

Operational exposure: The --api-key option does not protect /metrics, /health or /invocations, so put a reverse proxy in front that allowlists only user-facing endpoints.
Multi-node traffic: Node-to-node communication is insecure by default, so keep multi-node GPU clusters on an isolated network inside the EU region.
Hardware scope: Confirm that your chosen GPUs and drivers are supported by the installation matrix for your release.
Release cadence: Minor releases arrive about every two weeks, so pin a version and test each upgrade.
Cost drivers: Accelerator capacity and the engineers who run it are the main recurring costs.

Before you commit, confirm with your cloud or datacenter provider which physical regions hold your GPUs, backups and logs, because inference hardware is only one part of where data lives.

How hosted OpenAI residency and self-hosted vLLM compare on EU controls

This table compares hosted OpenAI EU residency with self-hosted vLLM across data location, approval, abuse monitoring, third-country access, version control and operations. Read each row as a question to put to OpenAI in writing and to your own team, since hosted terms come from your contract. The main takeaway is that self-hosting gives you direct control over location and retention, but it also gives you the work of evidencing those controls.

Control	Hosted OpenAI with EU residency	Self-hosted vLLM in your EU region
Data location	Confirm in writing which endpoints, stored data and model processing fall inside the EU region for your account.	Prompts, responses, logs and embeddings stay where you deploy the cluster, within your chosen cloud region or datacenter.
Sales approval	Residency requires approval through OpenAI sales, so confirm the approval terms, scope and any changes to them in writing.	No vendor approval is needed. vLLM is Apache-2.0 licensed, so the work is hardware, cloud contracts and internal sign-off.
Abuse monitoring	Confirm whether abuse monitoring reviews or retains prompts in the EU region, and which controls you can switch on or off.	You define logging and retention. Document what the server logs, who can read it and how long it is kept.
Third-country access exposure	Assess US legal reach over data OpenAI holds in its possession, custody or control, and confirm how it handles government requests.	Exposure depends on the jurisdiction of your cloud or datacenter operator and any remote support access they hold. Map each operator with access.
Processor and transfer terms	Review the data processing agreement, sub-processor list and transfer terms, and confirm which parties can access data for support.	If you host in the EU and no third-country party can reach the data, transfer questions narrow to your own operators and support paths.
Model version control	The vendor sets model versions and retirement dates, so confirm which snapshots are available in the EU region and for how long.	You pin the model weights and vLLM version. Each upgrade is a change you test, approve and can roll back.
Who operates the stack	OpenAI runs inference. Your team runs the integration, key management and the evidence your assessor requests.	Your team or an implementation partner runs GPUs, vLLM, patching, the reverse proxy, monitoring and on-call support.

How to deploy vLLM and a gateway inside an EU region

This sequence moves model serving into an EU region you choose, with processor terms confirmed first, so prompts, responses, logs and metrics stay inside infrastructure you control.

Choose the EU region and confirm processor terms
Your compliance lead and procurement team pick an EU region or EU datacenter and ask the cloud or datacenter provider, in writing, where compute, storage, backups, support access and logs are located, and which contractual terms govern processing. Ask whether any support or administrative access can happen from outside the EEA, because EDPB Guidelines 05/2021 treat remote access from a third country as a transfer. Confirm this with your provider and your data protection officer, since the answers depend on their contracts. The step is done when you hold a written location and access statement for every service in the stack.
Size GPU capacity for the target models
Your platform engineers estimate accelerator memory from the model sizes, context lengths and expected concurrency you plan to serve, and check that the chosen region actually offers those GPU types. vLLM supports NVIDIA CUDA, AMD ROCm and Intel XPU GPUs, as well as CPUs on several architectures, so confirm the hardware in your region against the vLLM installation documentation. Ask the provider for capacity commitments in writing, since GPU availability differs by region. The step is done when a sizing note records the chosen instance types, the model and the expected load, and the provider has confirmed the capacity exists in the EU region.
Deploy vLLM on Kubernetes
Your platform team deploys vLLM into an EU-hosted Kubernetes cluster using native manifests or Helm, following the options in the vLLM Kubernetes documentation. Pin the vLLM version and the model weights you have validated, and store the model artifacts in an EU bucket or volume. Pods run vllm serve with the OpenAI-compatible endpoints your applications need. The step is done when a test request to /v1/chat/completions succeeds from inside the cluster and the pod logs show the model loaded on the intended GPUs.
Put a reverse proxy in front of vLLM
Your security engineers place a reverse proxy in front of vLLM that allowlists only the /v1 endpoints that applications use. The vLLM security documentation states that --api-key protects /v1 routes but not /invocations, /pause, /resume, /abort_requests, /metrics or /health, and it warns against relying on that key alone. Block all other paths at the proxy, and expose /metrics only to your monitoring system on an internal address. The step is done when requests to unlisted paths return a refusal from outside the cluster and the /v1 endpoints still respond.
Isolate nodes for multi-node runs
If a model needs more than one node, your network engineers place all inference nodes on an isolated network segment. The vLLM security documentation states that communication between nodes is insecure by default, without authorization or encryption, so the segment must be protected from other workloads and from traffic outside the cluster. Keep Ray or the multiprocessing backend traffic inside that segment. The step is done when a network scan from a workload outside the segment cannot reach the node-to-node ports, and the cluster still serves requests.
Deploy LiteLLM with PostgreSQL and Redis
Your platform team runs the LiteLLM AI Gateway in the same EU region, pointed at the vLLM service as an OpenAI-compatible backend. LiteLLM names PostgreSQL as its primary supported database, which virtual keys require, and it requires Redis 7.0 or newer once more than one proxy instance runs, for shared rate limits and router state. Store the LITELLM_MASTER_KEY in a secret manager, and run one Uvicorn worker per pod. The step is done when two proxy replicas share limits and spend data, and a virtual key restricted to one model can call it while other models are refused.
Send logs and traces to an EU-hosted store
Your observability team configures LiteLLM logging and any tracing tool, such as a self-hosted Langfuse or Opik deployment, so that prompts, responses and metadata are written only to a store hosted in the EU region. Review what each log callback records, since full prompt bodies can contain personal data, and set retention to match your own policy. Confirm that any SaaS monitoring or alerting endpoint you keep is either excluded or located in the EU. The step is done when a test prompt appears in the EU store and no logging destination outside the region receives it.
Restrict egress so no prompt leaves the region
Your network and security teams apply egress rules to the inference cluster, the gateway and the logging stack so that they can reach only the internal services they need, plus any EU endpoints you have approved. Model downloads and container image pulls should come through an EU mirror or be completed before the cluster is locked down. Test with a deliberate outbound call to an external host and record the blocked attempt. The step is done when the egress test fails as designed, the gateway still serves requests, and the rules are documented for your auditors.

Questions EU teams ask before self-hosting an LLM

Does self-hosting an open-weight model remove GDPR processor duties?

No. Self-hosting removes a third-party processor for inference, but you remain the controller for every prompt, response and log that contains personal data. You still need a lawful basis, a record of processing, access controls, retention limits and a DPIA where the risk warrants one. Your cloud or datacenter provider may still be a processor for hosting, so review that contract too. The Article 44 transfer questions largely fall away when the data never leaves EU infrastructure, but you should confirm where backups, logs and observability data are stored, because EDPB Guidelines 05/2021 treat storage in a cloud outside the EEA as a transfer. Ask your counsel to confirm the position for your specific data flows.

What cost drivers decide the bill for self-hosted LLM inference?

The bill has four main components, and only some recur. GPU or accelerator capacity is the largest recurring line and depends on model size, concurrency and utilization. The gateway, monitoring stack, storage and backups are recurring infrastructure. Implementation, hardening and documentation are mostly one-time effort. The engineers who operate the stack, patch vLLM and handle on-call are the cost most often underestimated. Utilization decides the comparison: idle GPUs cost the same as busy ones. Ask any implementation partner to price each component separately, state which items are one-time and which recur, and name the assumptions for peak traffic and model choice.

How long does a pilot take and what does it need from our team?

The duration depends on scope, the number of models, your Kubernetes or cloud setup and how quickly approvals move, so no fixed timeline applies here. A typical pilot needs a named technical owner, access to a non-production cluster or GPU pool in your EU region, a decision on the models to test, sample prompts that reflect real workloads, and someone who can approve network and identity changes. Expect the first deliverable to be a working endpoint behind a gateway with authentication and logging. Agree the timeline and client responsibilities in the written scope before work starts, since most DeployOpen engagements are fixed-scope.

Can our in-house team run a self-hosted LLM stack alone?

Many teams can, provided they already run Kubernetes or GPU workloads and have people who understand model serving. The skills needed span GPU driver and scheduler management, vLLM tuning, gateway configuration, secrets handling, monitoring and incident response. The vLLM security documentation also warns that node-to-node communication is insecure by default and that the --api-key option does not protect every endpoint, so a reverse proxy with an allowlist is needed. Assess whether you have that coverage for weekends and on-call, and whether you need outside help for the initial build, hardening or a retainer for upgrades and testing.

Do the EU Data Act switching rights apply to our LLM service?

They may apply, but the answer depends on how your current contract classifies the service. The Data Act, applicable since 12 September 2025, gives customers of a data processing service the right to switch to another provider or to on-premises ICT infrastructure, with a maximum two-month notice period and a 30-day transition under Articles 25 and 29. Switching charges are banned from 12 January 2027. Whether a hosted model API falls within that definition is a question for your legal team, so read the contract and ask the provider directly. Self-hosting gives you a practical exit path regardless of the outcome.

How does a DPIA and transfer assessment change when we self-host?

The DPIA stays necessary where processing is high risk, but its focus shifts from the provider's transfer exposure to your own controls. A transfer assessment becomes simpler for prompts that stay in your EU environment, yet you still need to document where logs, backups and telemetry sit. Schrems II and the EDPB Recommendations 01/2020 remain relevant if any US-based vendor retains access, including through support tooling. The EU-US Data Privacy Framework adequacy decision is under appeal in Case C-703/25 P, so do not treat it as settled. Ask your data protection officer to review the revised data map before go-live.

What should a quote for a self-hosted LLM engagement include?

A useful quote lists the scope, deliverables, environments and acceptance criteria in writing before any work starts. It should separate one-time items such as design, implementation, hardening, data-flow documentation and knowledge transfer from recurring items such as support, upgrades and testing retainers. Ask for the GPU sizing assumptions, the models and versions covered, the gateway and monitoring components, the controls evidence prepared for your assessors, and the client responsibilities that affect timing. DeployOpen does not publish a price list, and a quote follows from the written scope. Clarify what is excluded, including GPU hardware purchases and cloud consumption charges.

Which EU-region options does the hosted OpenAI API offer for residency?

OpenAI's European data residency requires sales approval and involves abuse monitoring controls for regional processing, so eligibility and coverage should be confirmed directly with OpenAI. Ask which endpoints are covered, whether prompts and outputs are stored and for how long, and which subprocessors and support access paths apply. The answers determine whether the hosted option meets your residency requirement or whether self-hosting is the simpler route. Compare those written answers with your own data classification before choosing.

Evidence regulators and auditors ask for

A self-hosted model keeps more of the evidence under your control, but it does not write the evidence for you. The cards below cover the records that show where personal data is processed, who can reach it, which model answered each request, and how you judged third-country exposure. DeployOpen prepares this evidence with you; it does not issue audit reports or certifications, and independent auditors and certification bodies make those determinations.

Processing locations and sub-processors

Keep a register of every place personal data is stored, processed or accessed, including backups, logs, object storage and support access. For each entry, record the region, the operator and whether any remote administrator outside the EEA can view data on screen. The EDPB guidance treats remote access from a third country as a transfer, so support paths belong in the register. Confirm the register matches your actual cloud account and region settings.

Processor agreements and transfer assessments

Collect signed processor terms with every cloud, hosting and support vendor in the chain, and the transfer assessment for each one that touches a third country. Standard contractual clauses under Commission Implementing Decision (EU) 2021/914 are one tool, but they still need a documented assessment of local law. Ask each vendor which of its own sub-processors can reach data and under what legal process.

Network diagrams and egress rules

Provide a current diagram showing where the vLLM server, the gateway, the database and the logging stack sit, and the firewall or security group rules that limit outbound traffic. For a LiteLLM or Portkey deployment, the record should show whether any control plane or telemetry endpoint is reachable from the data plane. vLLM's security guidance says multi-node traffic is insecure by default and should run on an isolated network, so that isolation should appear in the diagram.

Gateway access and key records

Export logs of who created, rotated and revoked gateway keys, and which models each key could call. LiteLLM virtual keys, budgets and team assignments are stored in PostgreSQL, so backups of that database become part of the evidence set. Confirm that the master key is held in a secret manager and that access to the admin UI is restricted through SSO. Retention periods for these logs are a policy decision you must document and enforce.

Model version and change records

Record the exact model weights, serving engine version, container image digest and configuration used for each environment, along with the date each change was approved. vLLM ships minor releases about every two weeks, so an unrecorded upgrade can change behavior between two audits. Keep the change ticket, test results and rollback plan for every model or engine update.

Third-country access exposure assessment

Write your own assessment of whether any provider in the chain could be compelled to disclose data to a non-EU authority, and what safeguards apply. The EDPB has said that where a provider needs data in the clear and foreign powers exceed what is necessary and proportionate, no effective technical supplementary measure may exist. Document the legal basis for each conclusion and have counsel review it. This assessment is the controller's responsibility and cannot be delegated to the vendor.

When to stay with OpenAI behind your own gateway instead

Keeping OpenAI behind a self-hosted gateway is a valid choice when your approved EU residency terms cover every data class you send, in writing, and your organization accepts the remaining transfer exposure. Put the question to OpenAI's sales and contract team as a data-class question: which categories of prompt, completion, file and log content are processed and stored in the EU, which are still processed in the US, and which abuse monitoring retention applies to each. Only the categories covered by written terms should go through the API. In that setup, a self-hosted LiteLLM gateway with virtual keys, per-team budgets and logging gives you one place to enforce which applications may send which data classes, so the routing decision is enforced in code rather than left to developers.

Stay with the hosted API only if you can answer no to a harder question: could any prompt in scope be exposed to a third-country authority in a way your controller obligations cannot accept? Article 48 of the GDPR limits how a non-EU court or authority order can be enforced in the EU, and the EDPB's Recommendations 01/2020 say that where a provider must access data in the clear and foreign public authority powers exceed what is necessary and proportionate, no effective technical supplementary measure can be expected to fix it. If prompts contain health records, customer financial data, employee files or other high-sensitivity content, that exposure is the signal to move the workload to open-weight models on vLLM in your own EU region. Document that analysis as part of the transfer assessment, and have it reviewed by counsel and your data protection officer before any production traffic moves. Their sign-off, not this page, decides the outcome.

How DeployOpen works on this

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own cloud accounts, data centers or air-gapped networks. For EU residency work, that means the open-weight model serving stack, gateway and observability layers run where the client's own legal and data protection team has approved them.

How we scope

Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.
The written scope names the EU region or datacenter, the data flows that stay in it, and every third-country access path to rule out.
Scope also records which compliance evidence the client's auditors or data protection officer will need, without DeployOpen issuing any attestation.

How we build

We deploy vLLM behind a reverse proxy that allowlists only the user-facing endpoints, with the gateway and model weights kept inside the approved region.
We configure the gateway, virtual keys, budgets, logging and identity integration, and test that no request path calls an endpoint outside the agreed boundary.
Security testing covers the model endpoint, gateway and monitoring stack, including application and API testing and cloud and infrastructure checks.

How we hand over

Handover includes architecture diagrams, data flow maps, network and access configuration, and runbooks for operating the stack in your environment.
We document residency controls, subprocessor and hosting locations, and the evidence that supports your GDPR transfer and processor assessment.
Your team receives training on upgrades, key rotation, model version changes and incident response for the inference service.

How we support

Ongoing support, upgrades and continuous security testing are available as retainers after handover.
Retainers can cover tracking vLLM and gateway releases, testing them in your environment and scheduling changes that keep the stack current.
Quotes follow from the written scope. Ask for a breakdown of one-time setup, testing, documentation and recurring support so you can compare options.

Self-hosted OpenAI API alternative for EU data residency