Self-hosted Azure OpenAI alternative for EU data residency

Self-hosted Azure OpenAI alternative for EU data residency: the short answer

If prompts and completions contain personal data and you are an EU controller, check the Data Zone EU and Standard regional deployments first, because both keep processing in Europe under Microsoft's controls. Global deployments may process prompts in any Azure region, so they do not meet a residency requirement on their own. Ask Microsoft in writing which processing locations apply to each deployment type you plan to use, what storage and abuse monitoring retention applies, and under which conditions the EU Data Boundary could be extended, since those terms decide whether a third-country transfer occurs.

Even with an EU region, the provider is a US company, and that matters under GDPR Article 44 and the CJEU's reasoning in Schrems II. The US CLOUD Act provision at 18 U.S.C. 2713 requires providers to preserve or disclose data in their possession, custody or control, wherever it is located. EDPB Recommendations 01/2020 state that where a provider needs data in the clear and third-country powers go beyond what is necessary and proportionate, the EDPB cannot envision an effective technical supplementary measure. Only open-weight models served on vLLM, which is Apache-2.0 licensed, in an EU location you control, with no third-country provider in the request path, remove that dependency entirely. Your data protection officer and counsel should record this assessment before you decide.

The trade-off is operating effort. Your team, or a partner you contract, owns GPU capacity, EU hosting, vLLM upgrades (new minor releases arrive about every two weeks), network isolation for multi-node runs, a reverse proxy that exposes only user-facing endpoints, metrics, patching and on-call cover. Start with a data flow map that shows where prompts, logs and traces travel, then confirm which Azure deployment type, if any, meets your residency rule. Read the vLLM security documentation before exposing any endpoint, and compare the recurring operating cost with the cost of staying on a compliant Azure deployment.

What EU data residency requires of an LLM deployment

Each card below is a legal or technical checkpoint that decides where prompts, completions, logs and traces may sit. Read together, they show that the question is not only which region a cloud service runs in, but who can reach the data, from where, and under which foreign powers.

Transfer conditions under Article 44

GDPR Article 44 applies Chapter V to every transfer of personal data to a third country, including onward transfers. For an LLM stack, a prompt that contains customer data and is processed by a provider outside the EU or EEA is a transfer that needs a valid transfer mechanism. Check where every hop runs: the model endpoint, the gateway, the trace store and any support tooling. Infringements of Articles 44 to 49 carry fines of up to EUR 20 million or 4% of worldwide annual turnover under Article 83(5)(c).

Remote access and storage count as transfers

EDPB Guidelines 05/2021 (version 2.0) treat remote access from a third country as a transfer, even when a support engineer only views data on a screen. Storing data in a cloud outside the EEA is also a transfer. For LLM operations, this reaches vendor support sessions, debugging of traces in a foreign console and backups replicated abroad. Ask every provider which staff and sub-processors can see prompts, completions and logs, and from which countries.

Supplementary measures have limits

EDPB Recommendations 01/2020 state that when a cloud provider must access data in the clear and the destination's public authority powers go beyond what is necessary and proportionate, the EDPB cannot envision an effective technical supplementary measure. Encryption keys held by the provider do not solve this if the provider can read the data. Where prompts contain sensitive records, the practical question is whether the processing can be kept under keys and administrators you control, in an EU location you choose.

CLOUD Act and Article 48 exposure

The US CLOUD Act provision at 18 U.S.C. 2713 requires US electronic communication and remote computing service providers to preserve or disclose data in their possession, custody or control, wherever it is stored. GDPR Article 48 allows a third-country disclosure order to be recognized in the EU only when it rests on an international agreement, such as a mutual legal assistance treaty. For US-headquartered providers, ask how they respond to such orders, whether they notify customers, and whether a contractual commitment covers the EU location of your data.

Data Privacy Framework status

Commission Implementing Decision (EU) 2023/1795 finds adequate protection only for US organizations on the Data Privacy Framework List. An appeal against the General Court's September 2025 judgment upholding it is pending as Case C-703/25 P. If a contract relies on this framework, record the recipient's listing and the date you checked it, and plan a fallback transfer mechanism. Schrems II, decided in Case C-311/18, showed how quickly such adequacy findings can be challenged.

Where Azure OpenAI deployment types meet the constraint

Global deployments are the broadest option. Prompts may be processed in any Azure region, which gives Microsoft capacity flexibility but means processing is not fixed to the EU. The EU Data Boundary is a Microsoft commitment tied to EU-located processing, so for residency purposes assume it does not cover Global deployments until Microsoft states otherwise in writing.

Processing location: prompts and completions may be processed in any Azure region, so an EU customer cannot point to a single EU processing location.
Transfer analysis: EDPB Guidelines 05/2021 treat storage outside the EEA and remote access from a third country as transfers, so processing in a non-EU region needs a Chapter V basis under GDPR Article 44.
US-law exposure: under 18 U.S.C. 2713, a US provider must preserve or disclose data in its possession, custody or control wherever that data sits, so Microsoft remains in scope for disclosure demands whatever region is used.
Boundary coverage: confirm whether any EU Data Boundary statement applies to this deployment type, and which logs, abuse monitoring records and support data are included.

Before relying on a Global deployment for personal data, check the current region behavior and data handling terms in Microsoft documentation rather than assuming EU customers are covered.

Which open-source components meet the constraint, and at what trade-off

This table compares the open-source components that could keep prompts, completions and traces in an EU location you control, with their license and data-flow caveats. Read each row against your own data flows: a component meets the residency constraint only if every call it makes stays where you put it.

Component	Role in an EU deployment	License and trade-off	Data-flow check before use
vLLM	Serves open-weight models behind an OpenAI-compatible API on GPUs in your EU location	Apache-2.0 with no paid edition; cost is GPU capacity, infrastructure and operators	Confirm the node network is isolated and that only user-facing endpoints are exposed through a reverse proxy
LiteLLM AI Gateway	Single OpenAI-compatible endpoint with keys, budgets, rate limits and fallbacks	MIT except the enterprise directory; SSO, SAML and audit logs need LiteLLM Enterprise	Confirm which providers and logging targets are configured, since fallbacks can send traffic outside the EU
Langfuse	Self-hosted traces, evaluations and prompt management in your own database and object storage	MIT outside the ee directories; some advanced features need an enterprise license key	Confirm PostgreSQL, ClickHouse, Redis or Valkey and S3 storage all sit in the EU location
Opik	Self-hosted tracing, evaluations and prompt management from Comet	Apache-2.0 and the README states the full platform can be self-hosted without a commercial license	Confirm telemetry settings and that no trace data leaves your network during setup
Arize Phoenix	Self-hosted tracing and evaluation on OpenTelemetry and OpenInference	Elastic License 2.0 is not OSI-approved and bars offering the software as a managed service to third parties	Check the license with counsel before offering it internally as a shared service
Portkey hybrid deployment	Gateway runs in your VPC while the control plane stays in Portkey's VPC	Commercial; the gateway calls out over HTTPS to the vendor control plane	Confirm which control-plane traffic leaves the EU and whether it counts as a transfer under your assessment
LangSmith EU Cloud region	Hosted tracing with data kept in the EU region where the account signed up	Commercial; billing information stays in the US	Confirm the signup region, billing data handling and any support access from outside the EU

What vLLM needs in place before it can hold regulated prompts

Before any regulated prompt reaches a self-hosted vLLM server, the team must close the gap between what the server exposes and what users should reach. The --api-key option protects the /v1, /v2, /inference and /cohere route families, but it does not cover /invocations, /pause, /resume, /abort_requests, /metrics or /health. The vLLM security documentation warns against relying on the key alone and recommends a reverse proxy that allowlists only the routes meant for users. For an EU residency design, this proxy is the control that keeps prompts inside the boundary you chose: it should pass only the inference routes your applications call, deny everything else by default, and be the only path from user networks to the server. Confirm the allowlist with a test that requests each non-allowlisted route and records the refusal.

If the deployment spans several nodes, the inter-node traffic is insecure by default. The vLLM security documentation states that communication between nodes has no authorization or encryption, so anyone who can reach those ports can join or disrupt the cluster. Place all tensor and pipeline parallel nodes on an isolated network segment that carries no user traffic, and confirm the segment's routing and firewall rules in your own environment. The same residency logic applies here: a node reached over a shared network may leave the EU boundary you defined, so network placement is part of the data location decision, not just an operations detail.

The /metrics endpoint exposes Prometheus-format operational data, including request counts, token throughput, latency histograms and KV cache use. This data is not prompt content, but it reveals usage patterns and capacity that should stay internal. Scrape it from your monitoring network only, and keep it off the reverse proxy's public allowlist. If your monitoring stack ships metrics to a hosted service, that export is a data flow you must assess under the same residency rules as prompts. Ask your security team to document where each metrics stream goes, who can read the dashboards, and how long they are retained.

How to deploy self-hosted models under an EU residency rule

This sequence takes a workload from legal scoping to a cut-over in which prompts, completions and traces are processed and stored in an EU location you choose and control, with each step producing evidence you can show to counsel and your assessors.

Map data flows and confirm the legal basis
Counsel and the data protection lead list every prompt, completion, log, trace and support channel, including who can view them from outside the EU. Under GDPR Article 44, any transfer of personal data to a third country, including onward transfers, must meet Chapter V conditions, and EDPB Guidelines 05/2021 treat remote access from a third country, even for support screens, as a transfer. The output is a signed data flow register that states the lawful basis and names every party with access. Confirm with counsel how the Data Act switching rights, which allow moves to on-premises infrastructure with a 30-day transition, fit your existing provider contracts.
Choose an EU region and GPU supplier
Select the region, data center and accelerator supplier, then confirm in writing where the GPUs, storage, backups and operator access sit. vLLM runs on NVIDIA CUDA, AMD ROCm and Intel XPU GPUs, so the hardware choice should follow model size, throughput targets and your team's skills. Ask the cloud or colocation provider whether support staff, backup replicas or subprocessors outside the EU can reach the environment, and whether any third-country law could compel access. The output is a site and supplier decision record with the access and support answers attached.
Serve the model with vLLM behind a reverse proxy
Deploy the open-weight model with vllm serve on the chosen hardware, using native manifests, Helm, the vLLM production-stack or another option listed in the vLLM Kubernetes documentation. The vLLM security documentation warns that --api-key does not protect /invocations, /pause, /resume, /abort_requests, /metrics or /health, so place a reverse proxy in front that allowlists only the endpoints applications need. Multi-node runs must sit on an isolated network because inter-node traffic is insecure by default. The output is a served endpoint that returns a valid chat completion while the admin and metrics paths are unreachable from user networks.
Add LiteLLM for keys, budgets and limits
Deploy the self-hosted LiteLLM gateway in the same EU environment and register the vLLM endpoint as a backend. Issue virtual keys per application or team, then set budgets, rate limits and spend tracking. LiteLLM's MIT-licensed core covers these controls, while SSO, SAML, audit logs and multi-team management sit in its enterprise directory under a separate license, so confirm which features your organization needs before the design is final. The output is a test key that succeeds, a rejected key that fails cleanly, and a spend record for that test call.
Route traces to EU-hosted Langfuse or Opik
Deploy Langfuse or Opik in the EU environment so traces never leave the region. Langfuse production setups run on Kubernetes with Helm and depend on PostgreSQL, ClickHouse, Redis or Valkey, and S3 or blob storage, so provision each component in the EU and confirm backup locations. Opik is Apache-2.0 licensed and its README states the full platform can be self-hosted without a commercial license. Point LiteLLM's logging at the chosen tool. The output is a trace from a test request that appears in the EU instance and nowhere else, which you verify by checking the storage endpoints.
Validate in staging against your prompts
Run a staging environment that mirrors production, with the same proxy allowlist, keys, budgets and logging. Replay representative prompts from your own workloads, compare output quality with the current service, and check the Prometheus metrics that vLLM exposes at /metrics for latency, throughput and KV cache use. Confirm that the staging proxy rejects every endpoint that is not on the allowlist. Keep the test results, configuration files and network tests as evidence. The output is a signed acceptance record that names the model version, the gateway version and the tests passed.
Cut over applications by changing the base URL
Move applications one at a time by pointing their OpenAI client base URL at the LiteLLM gateway and using the new virtual keys. Because vLLM exposes OpenAI-compatible endpoints such as chat completions and embeddings, most application code changes only configuration, though you should still confirm each feature used, such as streaming or tool calls, against the staging results. Keep the previous endpoint available until the rollback window closes, then remove it. The output is a cut-over log per application, a confirmed rollback path, and an updated data flow register reflecting the new processing location.

Questions buyers ask before hiring help for an EU self-hosted LLM

What does a self-hosted EU LLM engagement cost, and what drives it?

The cost splits into four parts: GPU or accelerator capacity in an EU location you control, the surrounding infrastructure (gateway, observability, storage, backups), implementation work, and the staff who operate the system after go-live. GPU capacity is usually the largest recurring line and depends on model size, concurrency and utilization. DeployOpen does not publish prices, and a quote follows from a written scope that names the models, environments, integrations and evidence you need. Ask any provider to separate one-time build effort from recurring operations and capacity, and to state which parts you would own directly. vLLM itself is Apache-2.0 licensed with no paid edition, so license fees for the serving engine are not the cost driver.

How long does an EU self-hosted LLM project take?

The timeline follows the written scope. Scope drives it through the number of models, the number of applications that need to switch to the OpenAI-compatible endpoint, the amount of evaluation on your own prompts, the GPU procurement lead time in your environment, and the compliance evidence you need collected. DeployOpen agrees scope, deliverables and timelines in writing before work starts, so ask for the schedule as milestones tied to those deliverables rather than a single date. Hardware lead times and your internal approval cycles are often the longest items, so start those inventories early.

What must our team provide before the engagement starts?

Four inputs make the scope accurate. First, a data flow inventory listing which applications send which personal data to which endpoints, including logs, traces and backups. Second, legal review of your transfer analysis, since controllers own the GDPR position and DeployOpen does not provide legal advice. Third, access to GPU capacity or a plan to procure it in a location you choose, with network isolation for multi-node runs. Fourth, named staff who will own the platform, approve changes and respond to incidents. Missing any of these usually moves work into later phases and changes the quote.

Can our own team build and run this without outside help?

Yes, if you have engineers who can operate GPU servers, Kubernetes, a gateway, observability and backups, and who can own security configuration. vLLM is open source and its documentation covers serving, parallelism, metrics and Kubernetes deployment. The points most teams underestimate are securing the endpoints (the vLLM --api-key option does not cover every route, so the docs recommend a reverse proxy that allowlists user-facing endpoints), isolating nodes in multi-node runs, and keeping model versions and evaluations current. Outside help is most useful for a fixed-scope build, a hardening review, or a retainer for upgrades and testing.

Is Data Zone EU enough to keep our prompts in the EU?

Not by itself. Data Zone EU processes inside the EU Data Boundary, which Microsoft can extend, but you still depend on a US provider's contract, support access and any third-country legal exposure. The EDPB Guidelines 05/2021 treat remote access from a third country, even only displaying data for support, as a transfer. Under Schrems II and the EDPB Recommendations 01/2020, where a provider's access to data in the clear is driven by third-country powers that go beyond what is necessary and proportionate, the EDPB does not expect a technical supplementary measure to be effective. Serving models in an EU location you control removes that dependency, but your legal team should still confirm the position for each data category.

What do the EU Data Act switching rules mean if we leave our provider later?

Under the Data Act, applicable since 12 September 2025, customers can switch from a data processing service to another provider or to on-premises ICT infrastructure, with a maximum two-month notice period and a 30-day transition. Switching charges are banned from 12 January 2027. These rights are set out in Articles 25, 29 and 50 of Regulation (EU) 2023/2854. In practice, the rules help you exit a cloud service, but they do not produce a working replacement. Keep your prompts, evaluation sets, model configurations and gateway settings in files you control so that the move to self-hosted models can happen on your schedule.

Does DeployOpen issue audit reports or compliance certifications?

No. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies perform those assessments. Ask any provider which evidence it will produce, in what format, and who will review it, and make sure your auditor agrees with that plan before the build starts.

Evidence EU regulators and auditors ask for

Moving inference into an EU location you control shifts the question from where Microsoft processes prompts to what you can show about every transfer, endpoint, key and log. The cards below list the records that tend to be requested, and what each should prove. DeployOpen prepares this evidence, but supervisory authorities, auditors and certification bodies draw the conclusions.

Records of processing and locations

Your records of processing should name every system that touches prompts and completions, the country where each one runs, and who operates it. For a self-hosted stack that means the vLLM hosts, the gateway, the trace store and any object storage used for logs. Check that the record matches the actual deployment, not the planned one. Any copy of data in a cloud region outside the EEA, including backups, is a transfer that the record has to show. Under Article 44 of the GDPR, every transfer to a third country must meet Chapter V conditions, so a clear location list is the starting point.

Transfer impact assessments

Where a provider or support path still has third-country exposure, a transfer impact assessment documents the risk and the measures taken. Remote access from outside the EEA counts as a transfer, even when staff only view data on a screen for support. Record the legal reach of every vendor you keep in the chain, including any US-parented processor, and the government access requests it could receive. Under EDPB Recommendations 01/2020, technical measures may not be enough where a provider must see data in the clear, so the assessment should say so plainly. Confirm the current status of the Data Privacy Framework with counsel before relying on it.

Network and endpoint configuration

Auditors usually ask which endpoints accept traffic and from where. vLLM's own --api-key option does not cover every route, including /metrics, /health and /invocations, so the docs recommend a reverse proxy that allowlists only user-facing endpoints. Keep the configuration files, firewall rules and proxy allowlists under version control. For multi-node runs, record that node-to-node traffic is unauthenticated by default and sits on an isolated network. Your evidence should show the deployed state, captured from the running systems.

Gateway access and key records

The gateway is where authentication, budgets and attribution live, so its logs carry the evidence of who sent which request to which model. Keep records of virtual key issuance, rotation, revocation and the team each key belongs to. Store the key management procedure and show that old keys stopped working on the stated date. Access logs should be retained under a defined policy and reviewed by someone other than the administrator. Confirm which fields your gateway logs, since prompts may appear in logs and need the same protection as the data itself.

Trace and log retention settings

Tracing platforms often hold full prompts and completions, which makes their retention settings a primary audit item. Document the retention period set in each store, such as the trace database, object storage and metrics, and show the configuration that enforces it. Where a platform exports traces to a bucket, record the bucket location, access policy and lifecycle rule. Confirm that deletion requests reach backups and exports as well as the live database. Retention must match the purpose you stated in your records of processing, not the platform default.

Model and version inventory

An inventory lists each model, its exact weight version, the license that governs it, the hardware it runs on, and the systems that call it. This matters for EU residency because you can show that the weights and inference run where your records say they do. It also lets you answer questions about changes, such as when a version was replaced and what it was replaced by. vLLM ships new minor releases about every two weeks, so pin the vLLM version in your deployment and record each upgrade. Confirm the license terms of each open-weight model with your legal team.

When self-hosting does not remove the residency question

Running open-weight models on vLLM in an EU location you control removes the largest residency exposure, which is the prompt and completion path through a provider's global routing. It does not remove every route by which non-EU law or non-EU people can reach the data. Before you sign off the design, ask counsel to confirm three residual points: whether the vendor of your cloud account, colocation provider or hardware supplier is subject to foreign access law with extraterritorial reach, whether the support and remote administration model involves staff located outside the EEA, and whether any EU-US transfer mechanism you rely on is still stable. Under the EDPB's Guidelines 05/2021, remote access from a third country, even only displaying data on a screen for support, counts as a transfer, so a vendor engineer in a non-EEA office who opens a troubleshooting session on your inference cluster is within scope.

On the transfer mechanism, the Data Privacy Framework adequacy decision, Commission Implementing Decision (EU) 2023/1795, has an appeal pending as Case C-703/25 P. Its validity is therefore not settled, and it only covers US organizations on the Data Privacy Framework List. If your self-hosted stack still sends telemetry, logs, backups or support bundles to a US-headquartered SaaS tool, that flow needs its own transfer analysis, and the *Schrems II* history means you should not assume a future ruling will leave the current position unchanged. Ask each log, tracing and monitoring vendor for its Data Privacy Framework status and its standard contractual clauses position, and ask counsel how to document a fallback if the adequacy decision is set aside.

Finally, check the foreign government access question for the infrastructure itself. The CLOUD Act provision at 18 U.S.C. 2713 applies to providers within its reach regardless of where data sits, and EDPB Recommendations 01/2020 state that where a provider needs data in the clear and foreign authority powers exceed what is necessary and proportionate, no effective technical supplementary measure can be expected. Encryption with keys you hold, and a documented assessment of who holds administrative access to the hosts, are the controls to examine. Record these answers in your transfer impact assessment so that the residency claim you make to auditors and customers matches what the architecture actually does.

How DeployOpen builds an EU-resident LLM deployment

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure, including EU regions you choose. For an EU-resident model deployment, we start by fixing where prompts, completions, logs and traces may flow, then build and hand over a system that keeps them there.

How we scope

Map every place prompts, completions, traces and logs would be stored or accessed, including support access from outside the EU.
Agree the target EU locations, the model families, the gateway and the observability stack in a written scope.
Confirm with your counsel and privacy team which transfer and access questions the design must answer before build starts.

How we build

Deploy vLLM on GPU capacity in the EU location you select, behind a gateway that exposes only the endpoints your users need.
Set up self-hosted tracing and metrics so telemetry stays in the same EU boundary as the model traffic.
Test access controls, network isolation and data flows, and record the results as evidence for your own assessments.

How we hand over

Deliver architecture diagrams, data flow maps, configuration records and runbooks written for your platform and security teams.
Document which EU location holds each data store, who holds administrative access and how that access is logged.
Walk your operators through upgrades, model version pinning and incident response before the engagement closes.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer after handover.
Retainers can cover vLLM and gateway upgrades, re-running data flow checks after configuration changes and model version reviews.
DeployOpen prepares systems and evidence for compliance assessments, while independent auditors and certification bodies issue any formal opinion.

Self-hosted Azure OpenAI alternative for EU data residency