Ollama vs vLLM: when a local runner fits and when a GPU server does
Choose between these two by asking how many people will call the model at the same time and what hardware you already have. Ollama fits when a small group works with models on one machine. It is MIT-licensed, installs as a single binary on macOS, Windows, Linux and Docker, and manages model downloads for you. Its REST API on port 11434 is OpenAI-compatible, so simple tools can reuse it. Its concurrency defaults are conservative: OLLAMA_NUM_PARALLEL is 1 parallel request per model by default, so a shared team endpoint needs deliberate tuning and testing. Ollama's own FAQ states that the local API does not require authentication and binds to 127.0.0.1 by default, so any sharing across a network means you add the access control yourself.
vLLM fits when many concurrent users need steady throughput on GPUs. It is Apache-2.0 licensed, has no paid edition, and runs on NVIDIA CUDA, AMD ROCm and Intel XPU GPUs. vllm serve exposes OpenAI-compatible endpoints such as /v1/chat/completions and /v1/embeddings, and it scales a model across GPUs with tensor parallelism inside a node and pipeline parallelism across nodes. That capacity comes with operating work: you size and buy accelerators, run the serving stack, and handle multi-node networking. Its security documentation says inter-node traffic is insecure by default and must be isolated, and that --api-key does not cover /metrics, /health or several control endpoints, so you should confirm a reverse proxy allowlist before exposing it.
The decision turns on three questions: user count, hardware and security hardening needs. If the group is small, the hardware is a workstation or mixed machines, and the data stays on one host behind a gateway you control, Ollama is usually the lower-effort choice. If concurrency, GPU fleets and an OpenAI-style serving tier are the requirement, vLLM is built for that job, and the cost is mostly accelerator capacity plus the people who run it. Ask your team to write down the expected peak concurrent users, the GPU models you can actually source, and who will own the reverse proxy, TLS and monitoring. Then run a load test on your own hardware before committing, since published throughput figures depend heavily on model, quantization and request shape.
Which platform fits which team scenario
This table maps common team scenarios to the platform that fits better and the facts behind that choice. Read each row for the condition that would change the recommendation, since concurrency limits, hardware and security hardening drive most of the decision.
| Scenario | Better fit | Reason from the supplied facts | Condition that changes the answer |
|---|---|---|---|
| Developer laptop, one person experimenting with open models | Ollama | Ollama runs on macOS, Windows, Linux and Docker, stores models locally and binds to 127.0.0.1:11434 by default, so no server setup is needed. | If the laptop has no suitable GPU and the goal is high-throughput batch work, neither tool may suit; test with your own models first. |
| Shared server for a small team (a few developers, one GPU or CPU host) | Ollama, with reservations | Ollama's OLLAMA_NUM_PARALLEL defaults to 1 request per model, so concurrent teammates queue. Raising it and exposing the API requires a reverse proxy, since the local API has no authentication. | If several people send requests at once and latency matters, benchmark both tools on your hardware before committing to Ollama. |
| Multi-tenant internal API serving many applications | vLLM | vLLM exposes OpenAI-compatible endpoints such as /v1/chat/completions and /v1/embeddings, and its --api-key option is not enough alone, so a reverse proxy that allowlists user-facing endpoints is recommended. | If the API needs per-team keys, budgets or rate limits, a gateway layer such as LiteLLM may be needed in front of either runtime; confirm what each provides. |
| Model too large for one GPU | vLLM | vLLM uses tensor parallelism within a node and pipeline parallelism across nodes, and it supports multi-node runs through Ray or its own multiprocessing backend. | If the model fits on one device and concurrency is low, the extra cluster setup may not pay off. |
| GPU cluster with multi-node models | vLLM | Multi-node communication in vLLM is insecure by default, so nodes must sit on an isolated network. Kubernetes options include Helm, KubeRay and the production-stack. | If the cluster cannot be isolated at the network level, the security work may outweigh the throughput gain; confirm the network design first. |
| Mixed CPU, laptop and edge hardware | Ollama | Ollama runs on CPU hosts and common desktop operating systems, while vLLM supports NVIDIA CUDA, AMD ROCm, Intel XPU, x86, ARM and Apple silicon through its own install paths. | If you have a supported NVIDIA or AMD GPU server and need serving throughput, vLLM becomes the better choice. |
| Regulated environment needing audit trails and hardened endpoints | Either, after hardening work | Ollama states that local runs do not expose prompts to it, and cloud features can be disabled with OLLAMA_NO_CLOUD=1. vLLM exposes Prometheus metrics at /metrics, which must be allowlisted or protected. | Neither tool is a compliance product. Ask your assessor what evidence they expect, then confirm the authentication and logging you must add in front of either runtime. |
How Ollama and vLLM are built and what that means for operations
Ollama and vLLM expose similar-looking HTTP APIs but make different assumptions about who calls them, how many machines they span and how they are secured. The cards below cover the decisions an infrastructure team inherits from each design, and the checks to run before either one goes near production traffic.
Runtime model and concurrency
Ollama is a single Go binary with a llama.cpp backend, so one process manages model loading and requests on one host. Its defaults are conservative: one parallel request per model, controlled by OLLAMA_NUM_PARALLEL, with up to 512 queued requests set by OLLAMA_MAX_QUEUE. vLLM is a serving engine built around batching on GPUs. For a team, the question is whether your expected concurrency fits Ollama's queue model or needs vLLM's batching. Confirm the peak number of simultaneous users on your own hardware before choosing.
API surface and client compatibility
Both servers expose OpenAI-style interfaces, so client code can often point at either one. vLLM's vllm serve provides /v1/completions, /v1/chat/completions, /v1/responses and /v1/embeddings. Ollama provides a REST API on port 11434 with OpenAI compatibility plus Python and JavaScript SDKs. Compatibility is not the same as feature parity. Test the exact endpoints, streaming behavior and tool calling your applications use against each server before standardizing on one.
Bind address and authentication
Ollama binds to 127.0.0.1 on port 11434 by default, and its local API has no authentication. Exposing it means changing OLLAMA_HOST and putting a proxy in front of it. vLLM's --api-key option does not cover every route: it protects /v1, /v2, /inference and /cohere, but not /invocations, /pause, /resume, /abort_requests, /metrics or /health. The vLLM docs recommend a reverse proxy that allowlists only user-facing endpoints. Confirm which routes your network exposes and test each one from outside the trusted segment.
Multi-node communication
vLLM spreads a model across GPUs with tensor parallelism within a node and pipeline parallelism across nodes, and it uses Ray or its own multiprocessing backend for multi-node runs. The vLLM security docs state that node-to-node traffic is insecure by default, with no authorization or encryption. Protecting it means placing nodes on an isolated network. Ollama is designed around a single host, so it has no equivalent cluster traffic to isolate. If you plan multi-node serving, budget for network segmentation and review it with your security team.
Metrics and capacity signals
vLLM exposes Prometheus-format metrics at /metrics, covering request counts, token throughput, latency histograms and KV cache use, with configurable histogram buckets. These signals help you size GPU capacity and alert on saturation. Ollama's runtime is simpler to observe at the host level, but the decision about what to measure, and how to alert on queue depth and loaded models, is yours to design. Confirm whether your monitoring stack scrapes /metrics and whether that route is reachable only from internal monitoring hosts.
Kubernetes deployment options
vLLM publishes several Kubernetes routes, including native manifests, Helm, the vLLM production-stack, KServe, KubeRay and NVIDIA Dynamo, so a platform team can match the option to its existing cluster tooling. Ollama can run in Docker, which keeps the deployment shape simple on a single host. Choosing a Kubernetes path for vLLM means owning GPU scheduling, node pools, model storage and upgrades across releases that ship about every two weeks. Ask your platform team which option fits your cluster and who will track vLLM releases.
How Ollama and vLLM compare on the features that drive the decision
This table compares the features that most often decide between a local runner and a serving engine, from license and paid editions to authentication, metrics and multi-node security. Read each row as a question to confirm for your version and configuration, because several defaults change with releases and settings.
| Capability | Ollama | vLLM |
|---|---|---|
| License | MIT license for the runtime, per the Ollama GitHub repository. | Apache-2.0 license, per the vLLM GitHub repository. |
| Paid edition | Cloud model plans are paid (Pro, Max, Team, Enterprise). Running models on your own hardware is listed as unlimited on the pricing page. | No paid edition. Cost comes from accelerator capacity, surrounding infrastructure and the people who operate it. |
| API surface | REST API on port 11434 with OpenAI compatibility, plus Python and JavaScript SDKs. | OpenAI-compatible endpoints including chat completions, completions, responses, embeddings and audio transcription. |
| Hardware support | macOS, Windows, Linux and Docker, with llama.cpp as the backend. | NVIDIA CUDA, AMD ROCm and Intel XPU GPUs, x86, ARM, Apple silicon and IBM Z CPUs, plus third-party plugins. |
| Parallelism | Runs models on one machine. Multi-GPU and multi-node serving are not the design focus. | Tensor parallelism within a node, pipeline parallelism across nodes and data parallel for mixture-of-experts models. |
| Concurrency controls | OLLAMA_NUM_PARALLEL defaults to 1 per model, OLLAMA_MAX_LOADED_MODELS and OLLAMA_MAX_QUEUE defaults to 512. Confirm values for your version. | Continuous batching and KV cache sizing drive throughput. Confirm the scheduler settings for your model and GPU. |
| Authentication | Local API at localhost:11434 has no authentication. Protect it with a proxy such as Nginx. | --api-key protects /v1 and some routes only. Pair it with a reverse proxy that allowlists user-facing endpoints. |
| Metrics | Confirm which metrics and logs your Ollama version exposes before building dashboards. | Prometheus-format metrics at /metrics covering requests, token throughput, latency and KV cache use. |
| Kubernetes options | Run it in Docker or on a host. Confirm any supported orchestration pattern for your environment. | Native manifests, Helm, the production-stack, KServe, KubeRay, LeaderWorkerSet, llm-d and NVIDIA Dynamo. |
| Multi-node security defaults | Not applicable for single-machine use. Confirm network exposure if you run it on a shared server. | Node-to-node traffic is insecure by default with no authorization or encryption. Isolate nodes on a private network. |
What each platform costs in licensing and paid tiers
Ollama is released under the MIT license, so you can run it in your own infrastructure without a license fee. Ollama's pricing page states that running models on your own hardware is always unlimited. Its paid plans apply only to its hosted cloud models, which are a separate product from the local runtime. Those plans are listed as Pro at $20 per month ($200 per year), Max at $100 per month, Team at $500 per month and Enterprise at custom pricing. The Enterprise plan adds model access controls, cost budgets per user or API key, a dedicated support channel and custom security options on top of the Team features.
- MIT license: The runtime itself carries no license fee for local use.
- Unlimited local use: Ollama's pricing page says running models on your own hardware is always unlimited.
- Cloud plans are separate: Pro, Max, Team and Enterprise pay for hosted cloud inference, not for the local server.
- Enterprise pricing is custom: Ask Ollama for a written quote covering budgets, support scope and security options.
- Your own costs remain: Hardware, power, patching and staff time for the local server are paid outside Ollama.
Before you budget, confirm with Ollama which plan terms apply to your use and whether any cloud models will be used at all.
vLLM is licensed under Apache-2.0 and has no paid edition. Its cost comes from the accelerators it runs on, the surrounding infrastructure, the implementation work to deploy it and the people who operate it. Because there is no license invoice, the budget is driven by GPU or other accelerator capacity, which may be bought or rented, plus the engineering time to put it behind authentication, a reverse proxy and monitoring. Those costs recur as long as the service runs, while implementation is typically a one-time project cost.
- Apache-2.0 license: The engine carries no license fee for commercial use under its terms.
- No paid edition: There is no vendor tier to buy, so cost is not set by a plan.
- Accelerators drive most spend: GPU purchase or rental, and the number of nodes, determine the largest cost line.
- Operators are a recurring cost: Upgrades arrive about every two weeks in minor releases, and someone must test and apply them.
- Third-party hosting terms: If you use a managed Kubernetes platform or a hosted GPU provider, their pricing applies separately.
Ask any implementation partner or cloud provider for a quote that separates one-time setup from monthly hardware and support costs.
The open-source license of a project answers what you may use, modify and redistribute. A paid cloud plan or enterprise contract answers what a vendor will charge for hosting, support or added controls. The two are separate questions. Ollama's MIT license covers the local runtime, while its paid plans cover hosted cloud models. vLLM's Apache-2.0 license covers the engine, and it has no vendor tier at all. Neither license determines what your own infrastructure will cost.
- License covers the code: MIT and Apache-2.0 govern the software you run, not your hardware bill.
- Vendor plans cover services: Ollama's paid tiers price hosted cloud inference and related controls.
- Absence of a tier is not free operation: vLLM's lack of a paid edition still leaves GPU and staff costs.
- Read the terms you actually accept: Cloud plans carry their own terms, which may differ from the license of the code.
- Map each cost to an owner: Assign hardware, support, cloud subscription and staff time to a budget line.
Confirm with each vendor which terms govern your deployment before you commit budget.
What drives the running cost of each platform
The running cost of Ollama and vLLM is mostly hardware and people, and the two platforms spread those costs differently. Ollama's Ollama pricing page states that running models on your own hardware is always unlimited, so the spend is the machine, its power and hosting, and the staff time for setup and upgrades. vLLM has no paid edition, so its cost is GPU or other accelerator capacity, the surrounding infrastructure, implementation and the people who operate it. Setup, hardening and integration are mostly one-time costs. Hardware, power, hosting, upgrades and any support retainer recur every year you run the service.
For vLLM, the largest cost drivers are accelerator count and deployment topology. Tensor parallelism within a node and pipeline parallelism across nodes mean a large model may need several GPUs, and multi-node runs need an isolated network because the vLLM security docs describe node-to-node communication as insecure by default. Hardening also adds reverse proxy work, since --api-key does not cover /metrics or /health. vLLM ships new minor releases about every two weeks, with v0.31.0 on October 5, 2026, so each upgrade carries recurring regression testing on your own GPUs. Confirm your release cadence and who owns that testing before you commit to a budget.
For Ollama, concurrency is the cost lever. The default OLLAMA_NUM_PARALLEL is 1 parallel request per model, so serving many simultaneous users usually means more hardware or more instances, and you should size that before comparing invoices. Ollama's Enterprise plan adds model access controls, cost budgets and a dedicated support channel, which is the paid support path to confirm with the vendor. Ask any quote to itemize the hardware specification and count, power and hosting, who performs hardening and upgrades, how often upgrades are included, whether support is a retainer or fixed scope, and what changes as user count grows. DeployOpen works from a written fixed scope, with ongoing support and upgrades available as retainers.
Moving from Ollama to vLLM or back, step by step
This sequence gives you a tested order for moving client traffic between Ollama and vLLM, so each cutover keeps working APIs, documented access control and a clear rollback path.
Inventory models, clients and API calls
Start by listing every model served today, its format and quantization, and every application that calls the runtime. Record which clients use the native Ollama API, which use its OpenAI-compatible surface, and which call the server directly from user devices. Owner: the platform team with each application owner. Done when you have one table mapping each client to its endpoint, model name, expected concurrency and the users it serves. Ask each team whether any client depends on Ollama-specific behavior such as its model management calls, which will need rewriting rather than repointing.
Confirm model format and hardware support
Check that each model you plan to move is available in a format the target runtime loads, and that the target hardware is on vLLM's supported list for your accelerators or CPU architecture. vLLM supports NVIDIA CUDA, AMD ROCm, Intel XPU, several CPU architectures and Apple silicon, and it runs on further accelerators only through third-party plugins. Confirm the exact model, quantization and GPU memory with a test load on the target machine. Done when a test load of each priority model completes and returns a response to a sample prompt. Models that fail this check stay on the current runtime for now.
Point OpenAI-style clients at the new endpoint
Move clients that already use OpenAI-compatible calls by changing the base URL and model name in configuration, not in application logic. vLLM's server, started with the vllm serve command, exposes chat completions, completions, responses and embeddings endpoints. Test each client against a staging instance and compare output format, streaming behavior and error handling. Owner: application teams with platform review. Done when every OpenAI-style client passes its regression tests against the new endpoint. Clients that used Ollama's native API need a separate rewrite and remain on the old runtime until that work is done.
Add a reverse proxy with an endpoint allowlist
Place a reverse proxy in front of the new runtime and allow only the endpoints that users need. The vLLM security documentation says the --api-key option does not cover every route, including metrics, health, pause and resume, so it should not be the only control. Configure the proxy to pass the paths your clients use and block the rest, and terminate TLS there. Done when a request to an unlisted path returns a refusal from the proxy, and a request to an allowed path succeeds with a valid key. Confirm the exact behavior with your security team and the vLLM security page.
Rebuild access control, metrics and identity
Access control does not carry over from Ollama, which by default binds to 127.0.0.1 and does not require authentication on its local API. Set up per-application keys or a gateway that issues them, decide how keys are rotated and revoked, and connect identity where your policy requires it. Enable vLLM's Prometheus metrics at /metrics and route them to your monitoring stack, restricted to internal scrapers. Done when a revoked key is rejected, and dashboards show request counts, token throughput and latency for each model.
Rebuild deployment manifests and capacity plans
Ollama's concurrency settings, including parallel requests per model, loaded model limits and queue size, do not map directly onto vLLM. Write new deployment manifests or service definitions for the target, using your chosen orchestration path such as native manifests or Helm, and size GPU memory and parallelism from a load test rather than from the Ollama defaults. For multi-node runs, place nodes on an isolated network because vLLM's inter-node communication is insecure by default. Done when a load test on the target shows the throughput and latency you need, and the manifests deploy from version control.
Cut over traffic and keep the old runtime ready
Shift traffic in stages, one application at a time, while keeping the previous runtime and its model files available. Watch error rates, latency and output quality on the new path for an agreed period set by your team. Keep the rollback step written down: which configuration to revert, and who approves it. Done when every client runs on the target, error rates match the baseline, and the rollback steps have been tested in staging. Moving back to Ollama follows the same sequence in reverse, with the Ollama model directory and its OLLAMA_HOST binding confirmed before traffic returns.
Retire the old runtime and record the change
After the rollback window closes, stop the old runtime, remove its service definitions and archive its model files according to your retention policy. Update architecture diagrams, runbooks, access records and the capacity plan so they reflect the new runtime. Done when no client resolves to the old endpoint, the old service is absent from monitoring, and the change record names the owner for the new platform. If you plan to keep Ollama for developer laptops, document that scope separately so it does not return to shared infrastructure by accident.
Where implementation projects for each platform usually run long
Ollama projects rarely run long on installation. They run long on the decisions around it. The binary installs quickly on macOS, Windows, Linux or Docker, but the local API listens on 127.0.0.1:11434 by default and has no authentication, so anyone who needs access from other machines requires a bind address change through OLLAMA_HOST and a reverse proxy such as Nginx in front of it. That proxy, and the access control on it, is usually the first task that stalls, because someone must own it. Model storage is the second delay point: models live under the default path for each operating system unless OLLAMA_MODELS moves them, and a team that leaves this unset on a server with a small system disk will discover the problem only after pulls start failing. Concurrency is the third. With OLLAMA_NUM_PARALLEL defaulting to 1 per model, OLLAMA_MAX_LOADED_MODELS and OLLAMA_MAX_QUEUE set the rest, and load tests often show queueing that the team did not plan for. Confirm the expected number of concurrent users before sizing these settings.
vLLM projects usually stall earlier, before any model is served. GPU driver and hardware validation across NVIDIA CUDA, AMD ROCm or Intel XPU is the first gate, and it needs hardware access that the client must provide, often through a change window. Parallelism configuration follows: tensor parallelism inside a node and pipeline parallelism across nodes must match the GPU layout, and multi-node runs need Ray or the built-in multiprocessing backend. The vLLM security documentation states that node-to-node traffic is insecure by default, so the network design must place those nodes on an isolated segment before go-live, which is a decision for the client's network team. The reverse proxy allowlist is the next delay, because --api-key does not cover /metrics, /health or /invocations, so someone must decide which paths users reach. Metrics collection and the Kubernetes deployment choice (native manifests, Helm, KServe, KubeRay and others) are further decisions. Name a single accountable operations owner and an identity approach for API keys before scoping starts.
Questions buyers ask before choosing between Ollama and vLLM
How much effort does it take to run Ollama?
Ollama takes the least effort of the two. You install the binary on macOS, Windows, Linux or Docker, pull a model and call the REST API on port 11434. Most ongoing work is configuration: setting OLLAMA_HOST if other machines must reach the server, tuning OLLAMA_NUM_PARALLEL (default 1 per model) and OLLAMA_MAX_QUEUE (default 512) for concurrency, and placing a proxy in front of it, since the local API does not require authentication. Your team also owns model storage, upgrades to frequent releases and capacity planning. Effort rises with user count and with the controls your security policy requires, not with the install itself.
How much effort does it take to run vLLM in production?
vLLM takes considerably more effort to run as a production service. It is a GPU serving engine, so the work includes the accelerator stack (NVIDIA CUDA, AMD ROCm or Intel XPU for GPU serving), choosing tensor or pipeline parallelism for models that span devices, and operating the API layer around it. Multi-node runs use Ray or vLLM's own multiprocessing backend. Kubernetes teams can choose from native manifests, Helm, the vLLM production-stack, KServe and other options. Upgrades are recurring work, because vLLM ships new minor releases about every two weeks. Effort grows with model size, cluster size and the number of teams sharing the endpoint.
Can an in-house team run vLLM without outside help?
Yes, if the team already runs GPU servers, containers and an inference stack, and has time to track a fast release cadence. A team without GPU operations experience usually needs help with driver compatibility, parallelism settings, monitoring through the /metrics endpoint and access controls. Ollama is easier to hand to a generalist because it has fewer moving parts, although a shared service still needs its own proxy, logging and patching. A fixed-scope engagement can cover the initial build, with deliverables agreed in writing first, and a retainer can cover upgrades and continuous testing afterward. Ask any provider which parts of the work they would hand back to your team.
How long does a typical Ollama or vLLM setup take?
The research available for this page does not give a setup duration for either tool, so do not accept a single number without a written scope. Time depends on the drivers that differ between the two. For Ollama, they are model download size, network speed and how many users need access. For vLLM, they are GPU driver and container work, model weight downloads, the parallelism layout, the proxy and authentication layer and validation under load. Ask for milestones, the environment assumed and what is excluded. Ask the provider to state which tasks sit in its scope and which stay with your staff, and ask how long the first upgrade cycle takes, since that work recurs.
What does the work need from us: hardware, network, identity and operators?
The requirements differ mainly in hardware and in the network and identity controls around the endpoint. Confirm each point against your own environment before you size the work:
- Hardware: Ollama runs on laptops, workstations, servers and Docker, including CPU-only setups. vLLM's GPU serving path needs NVIDIA CUDA, AMD ROCm or Intel XPU hardware, and it also runs on several CPU architectures.
- Network: keep Ollama bound to 127.0.0.1 or publish it only through a reverse proxy. Isolate vLLM's inter-node traffic on a private network, because its security docs say that traffic is insecure by default.
- Identity: Ollama's local API has no authentication, so your proxy or gateway must supply it. vLLM's
--api-keyoption does not cover every endpoint, so allowlist user-facing routes at a reverse proxy. - Operators: name owners for patching, metrics, model lifecycle and on-call before the build starts.
Can Ollama be locked to local models only?
Yes, but you must set and verify it in your own environment. Ollama's FAQ says that local runs do not expose your prompts and that cloud features can be turned off with OLLAMA_NO_CLOUD=1 or in the server configuration file. Its pricing page describes running models on your own hardware as unlimited, so local use does not depend on a paid plan. Direct cloud inference uses an API key sent as a Bearer token, so an installation that never receives a key cannot reach that path, but do not rely on that alone. Block outbound traffic to Ollama's cloud endpoints at the firewall, then test that a local model still answers and that cloud requests fail.
Does vLLM have a paid edition or a license fee?
Neither tool requires a license fee for self-hosting. vLLM is Apache-2.0 licensed and has no paid edition, so its cost comes from accelerator capacity, the surrounding infrastructure, implementation and the people who operate it. Ollama's pricing page says that running models on your own hardware is always unlimited, and its paid plans apply to hosted cloud models: Pro at $20 per month ($200 per year), Max at $100 per month, Team at $500 per month and Enterprise at custom pricing. For either tool, GPU hardware, power, staff time and any outside implementation are the main costs. Ask for a quote that separates one-time build work from recurring upgrades, monitoring and support.
What security hardening does vLLM need before users connect?
vLLM's --api-key option is a starting point, not a complete control. Its security docs say the option protects the /v1, /v2, /inference and /cohere routes but not /invocations, /pause, /resume, /abort_requests, /metrics or /health, and they warn against relying on it alone. The docs recommend a reverse proxy that allowlists only the endpoints meant for users, and that proxy is the natural place for TLS, authentication and access logging. Restrict the metrics endpoint as well, since it exposes request and latency data. Ollama's local API needs the same proxy treatment because it has no authentication by default. Test the allowlist from outside the network and confirm that every route you did not intend to expose refuses requests.
How DeployOpen helps teams choose and run either platform
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For Ollama vs vLLM, we start by agreeing on the user count, hardware and security requirements, then build and hand over only the option that fits the written scope.
How we scope
- Confirm expected concurrent users, peak request patterns and the GPU or CPU hardware already available.
- Define which endpoints are exposed to users and which stay internal before any build starts.
- Agree the scope, deliverables and timelines in writing; most engagements are fixed-scope.
How we build
- Install and configure the chosen runtime in your environment, including bind addresses and model storage paths.
- Place multi-node vLLM deployments on an isolated network and put a reverse proxy in front of user-facing endpoints.
- Test concurrency limits, authentication behavior and metrics collection against the requirements agreed in scope.
How we hand over
- Document the configuration, network layout, access controls and the endpoints intentionally exposed to users.
- Provide runbooks for model updates, release upgrades and capacity changes on your own hardware.
- Walk your platform and security teams through the handover so they can operate the system without us.
How we support
- Offer ongoing support, upgrades and continuous security testing as a retainer after handover.
- Re-test the proxy allowlist and authentication settings after each upgrade to the serving stack.
- Review capacity and queue settings as user counts grow, and confirm changes against your own benchmarks.
