What a private LLM chat cost really depends on, and how to compare it
The inference and chat software behind a private assistant is largely free to license, so the total cost is driven by GPUs or cloud GPU capacity, integration and security work, and the people who operate the service. The right answer depends on three variables: how many people use the assistant at the same time, how large the models must be to meet your quality bar, and whether your team or a partner runs it day to day. A CISO or CIO comparing this with per-seat assistant subscriptions or per-token APIs should price all of those components, not the software line alone. Start with the components that recur. Hardware or cloud GPU capacity is the largest variable and is sized to concurrency, not headcount. Ollama, which runs on your own hardware, is free to use in that setting, and its paid cloud plans apply only if you choose cloud-hosted models. vLLM has no paid edition, so its cost sits entirely in accelerator capacity, the surrounding infrastructure and operations. Open WebUI, the chat interface, is licensed under the Open WebUI License, which restricts removing branding above 50 users in any rolling 30 day period unless you hold an enterprise license, so confirm whether that license is needed for your user count and whether it is priced per seat or per deployment.
Compare that with the one-time and recurring work around the platform. Single sign-on and group provisioning, PostgreSQL and Redis for multiple replicas, retention and logging design, and the security review of the inference endpoints are mostly one-time engineering, with ongoing patching and upgrades that recur because these releases move quickly. Ask any implementation partner for a fixed-scope quote from written scope, with separate lines for hardware sizing, integration, security testing, operations and support, so you can see what is one-time and what is a retainer. DeployOpen prepares that quote from the written scope and does not publish prices, so the comparison you can make is between your own staffing estimate and the scoped partner proposal, both measured over the same period.
Which cost components sit in-house, with a partner and in SaaS
This table shows, for each cost component of a private chat assistant, who pays in each model and whether the cost is one-time or recurring. Read each row across to see where the bill lands, then check the quote against the same rows. The main takeaway is that the software licences are often the smallest line, while GPUs, identity integration, security testing and day-to-day operations carry most of the ongoing spend.
| Cost component | In-house | Partner | SaaS |
|---|---|---|---|
| GPU servers or cloud GPUs | Your team buys or rents GPU capacity; purchase is one-time, hosting and power recur | Partner may size and procure hardware, usually billed as a pass-through plus its fee; you still own the capacity | Provider supplies capacity inside its per-seat or per-token price; recurring and bundled |
| Inference software (Ollama, vLLM) | Ollama local use is listed as unlimited on its pricing page; vLLM has no paid edition, so cost is staff time and hardware | Partner configures and tunes serving; quote the engineering hours, recurring for upgrades | Not applicable; model serving is hidden behind the subscription |
| Chat interface (Open WebUI, LibreChat) | Open-source core is free to run; your team deploys and maintains it | Partner deploys, customizes and documents it under a fixed-scope agreement, usually one-time | Included in the subscription; recurring fee per seat |
| Enterprise license (Open WebUI) | Needed only for white-labeling or branding above 50 users in a rolling 30-day period; pricing is quoted after contacting enterprise sales with an expected seat count | Partner can help you assess whether the license applies; the license fee itself is paid to the vendor | Not applicable; the provider's terms govern branding and features |
| Identity and SSO integration | Your identity team pays for OIDC, LDAP and SCIM setup and testing, usually one-time with ongoing group changes | Partner builds and tests the connection; one-time fee plus support if you retain it | Provider configures SSO, often on its standard tiers; confirm enterprise identity features are included |
| Data stores for multiple replicas (PostgreSQL, Redis, shared storage) | Your platform team provisions and backs up PostgreSQL, Redis and shared file storage, recurring infrastructure cost | Partner sizes and deploys these services; one-time build plus recurring hosting in your accounts | Provider hosts storage; check where chats and files are stored and how long they remain |
| Security testing and hardening | Your security team pays for reverse proxy allowlisting, network isolation and testing; often one-time, repeated after upgrades | Partner runs hardening and testing as fixed-scope work, with optional retainer for continuous testing | Provider's security reports are available on request; your own testing of the tenant is still your cost |
| Logging and retention | Your team pays for log storage, retention policy and access controls, recurring with volume | Partner designs log pipelines and retention settings; one-time build plus recurring storage in your environment | Provider's default retention applies; confirm export and deletion terms before signing |
| Model updates | Your team tests and rolls out new model and release versions, recurring staff time | Partner runs update cycles under a retainer, which is recurring; confirm the scope of each release | Provider updates models on its schedule; you may have limited control over timing |
| Day-to-day operations | Your staff pays for monitoring, incident response, user support and capacity planning, recurring | Partner provides support tiers as a recurring retainer; confirm response expectations in writing | Included in the subscription; support levels vary by plan |
How build, partner and SaaS shift the cost over time
Building in-house means your own engineers select, deploy and run Open WebUI or LibreChat with Ollama or vLLM. The software itself carries no subscription, so the largest costs are GPU capacity and staff time, both paid to your own people and suppliers.
- Engineering: Your platform team owns GPU node builds, the Open WebUI deployment, PostgreSQL and Redis for multiple replicas, and SSO wiring. Budget for the hours this takes beyond the pilot.
- On-call: Inference outages, queue backlogs and GPU faults land on your rota. Ollama allows one parallel request per model by default, so capacity planning becomes part of the on-call playbook.
- Upgrade cadence: Ollama v0.40.5 was released on October 14, 2026, and vLLM ships new minor releases about every two weeks. Someone on your team must test and roll out these changes continuously.
- License terms: Ollama and vLLM are licensed under MIT and Apache-2.0. Open WebUI is distributed under the Open WebUI License, which restricts removing or altering branding above 50 users in a rolling 30-day period unless you hold an enterprise license or written permission.
- What you keep: Prompts, files, chat history, model weights and logs stay in your infrastructure, and you set the retention rules.
Before committing, estimate staff hours and GPU lead times in your own environment, and confirm each license against your expected user count.
An implementation partner plans, builds, hardens and documents the same stack in your environment, then operates it under a written scope. The initial build is bought as a defined project, and ongoing support or upgrades can be bought as a retainer, so the budget shifts from open-ended hiring toward agreed deliverables.
- Engineering: The partner carries the initial build, SSO and group mapping, vector store and gateway integration under a statement of work. Scope, deliverables and timelines should be agreed in writing before work starts.
- On-call: Who answers pages after go-live is a contract term. Confirm response expectations, escalation paths and whether GPU hardware support is included.
- Upgrade cadence: A retainer can cover testing and rollout of new Ollama, vLLM and Open WebUI versions. Ask how the partner validates a model change before it reaches staff.
- License terms: Software licenses stay with the publisher, and upstream terms still apply, including Open WebUI branding limits. Ask the partner to list which licenses it has assessed and which need a direct agreement with the vendor.
- What you keep: Data, models, logs and encryption keys should remain in your accounts, with partner access granted and revoked through your own identity controls. Put that in the contract.
Ask for the written scope, the support terms for each service, and an exit plan that hands over runbooks and configuration.
A commercial assistant or per-seat subscription moves engineering, hosting and upgrades to the vendor, and turns cost into a recurring fee per user or per usage unit. It is usually the fastest route to staff access, but the organization gives up direct control over where prompts are processed and how long they are kept.
- Engineering: The vendor runs the platform, so your team's work shifts to configuration, identity integration and policy. Check which SSO and group features each plan includes.
- On-call: Outages are handled under the vendor's support tier. Ask what support is included at each price level and whether incident notices reach your security team.
- Upgrade cadence: The vendor decides when models and features change. Usage-based cloud plans show the pricing pattern: Ollama lists Pro at $20 per month ($200 per year), Max at $100 per month, Team at $500 per month and Enterprise at custom pricing on its pricing page. Its Enterprise tier adds per-user or per-key cost budgets and model access controls.
- License terms: You accept subscription terms, usage limits and data processing terms. Confirm retention periods, whether prompts are used for training, the subprocessor list and where data is processed.
- What you keep: Chat history and uploaded files sit in vendor storage, so log export for audit and data return at contract end must be written into the agreement.
Model the recurring cost at your expected seat count and usage volume, and confirm the data processing terms before staff upload any files.
What is one-time and what recurs every month
The one-time costs of a private chat assistant are the work that happens before and during rollout: architecture, SSO and group integration, initial security testing, migration of knowledge sources into the platform, rollout planning and user training. Each of these is billed as project effort against a written scope, so the quote should list them as separate line items. Ask the partner to name the identity provider and protocol (OIDC, LDAP or SCIM provisioning), the database and shared storage for multiple replicas, and whether the security test covers the chat interface and the model endpoints separately. Open WebUI's scaling guide requires PostgreSQL rather than SQLite for multi-instance deployments and Redis plus shared file storage for multiple replicas, so those components belong in the one-time build and in the recurring hosting estimate.
The recurring costs are the ones that continue after launch. The largest is compute: GPU servers you own, which add power, cooling and data center space if they sit on premises, or cloud GPUs billed by the hour or month. Next come support contracts or retainers, which cover patching, incident response and model changes. Upgrades are a standing cost, not an occasional one. vLLM shipped new minor releases roughly every two weeks in 2026 (v0.28.0 on August 26 through v0.31.0 on October 5), and Open WebUI published v0.12.3 on October 16, 2026 after v0.12.0 on October 10. Each release needs testing against your SSO, groups and inference endpoints before it reaches users. Ask the partner how many upgrades per year the retainer includes and who signs off on them.
Logs add a recurring storage cost that many estimates leave out. Where the EU AI Act applies, Article 19(1) requires providers to keep the automatically generated logs for at least six months unless other law sets a longer period, and Article 12(1) requires the system to allow automatic event recording over its lifetime. Whether your assistant falls under those articles depends on your use and role, so confirm it with counsel first. Once confirmed, budget for log storage, retention management and periodic log reviews, and ask any quote to state the retention period, the storage location and who performs the reviews. Enterprise add-ons such as the Open WebUI license or Ollama's paid cloud tiers are separate recurring items to confirm in writing.
Which factors push the cost up or down
Most of the cost of a private LLM chat sits in a few decisions: how many people use it at once, which models run, what hardware serves them and how deeply the assistant connects to identity and data. Each card below names the direction of the effect and what to confirm in your own environment before you compare a build, partner or SaaS quote.
Concurrency and peak users
Concurrency sets how much GPU capacity you buy or rent. Ollama processes one parallel request per model by default, controlled by OLLAMA_NUM_PARALLEL, so a busy peak can queue or need more hardware than average use suggests. Higher peak concurrency pushes cost up in GPUs and in the operations needed to run them. Confirm your peak simultaneous users, not your total headcount, and whether the business expects bursts at fixed times such as month-end.
Model size and model count
Larger models need more GPU memory and more accelerators per instance, so each step up in model size raises hardware cost quickly. Running several models for different tasks multiplies the memory footprint, even when each is used lightly. Smaller models served on fewer GPUs lower cost but may lower answer quality for complex work. Ask the partner which models they propose, the memory each needs at your context length, and whether one general model can cover most requests.
GPU type and capacity ownership
Owned GPUs carry a large up-front purchase and recurring power, cooling, hosting and replacement costs, but a steady workload can make the unit cost low over time. Rented cloud GPUs move cost into a recurring bill that scales with hours used, which suits pilots and uneven demand. Hardware generation and memory per card also change how many cards a model needs. Confirm the accelerator vendors your serving engine supports, the lead time for hardware, and whether capacity must sit in a specific region or facility.
Open WebUI replicas and shared services
Each extra Open WebUI replica adds compute, and multi-instance operation requires PostgreSQL instead of SQLite, Redis for shared session and application state, and shared file storage so uploads and knowledge remain reachable from every replica. These services are recurring costs to run, back up and patch. Ask whether your chosen vector database is included or separate, and who owns backup and restore for the database and file store.
SSO, SCIM and knowledge source integration
Connecting the assistant to OIDC or LDAP sign-in, SCIM 2.0 provisioning, group-based roles and internal document sources adds design, configuration and testing work. Each additional knowledge source, such as a file share, wiki or ticketing system, adds connectors, access rules and ongoing maintenance. Deeper integration usually raises the one-time implementation fee and the long-term support burden, but it reduces manual user administration. Confirm which identity provider you use, how groups map to roles, and which sources must respect existing document permissions.
Security and compliance evidence
Logging, retention, access reviews, endpoint protection and records for model changes all need to be built and kept. The vLLM security guidance says its --api-key option does not cover every endpoint, so a reverse proxy allowlist is usually needed, and that adds design and testing effort. Where the EU AI Act applies, event logs must be kept for at least six months, which sets a storage and retention requirement. Ask the partner for the evidence pack they will produce, who keeps it and how long it is retained.
When per-seat subscriptions and per-token APIs cost more
Per-seat subscriptions cost more as headcount grows, whether or not people use the tool. A seat fee is paid for every licensed user, so a large group of occasional users can cost as much as a small group of heavy ones. Per-token API pricing reverses that pattern: the bill follows usage, including prompt length, the size of pasted documents and the volume of files sent for analysis. A short prompt from a few people may be cheap, while a team that routinely uploads long contracts or reports can see the spend climb quickly. Before comparing, pull the seat count, the number of active users per week, and the token or request volume from any pilot or existing API logs, and split the usage by team and by use case.
A self-hosted model behaves differently. Once the server is bought or rented, the cost of a single prompt is driven mainly by how busy the hardware is, not by how many tokens are processed. Ollama's pricing page states that running models on your own hardware is always unlimited, which means the software side adds no per-query fee; the cost sits in GPU or server capacity, the people who operate it and the time spent on upgrades. Heavy users therefore move the cost question into capacity planning: the team must decide how many concurrent requests a model must serve at peak, and whether the hardware stays idle at night. Ask a partner to model peak and average concurrency from your usage data rather than quoting a single figure.
The comparison also has a data-location dimension. When models run locally, prompts, uploaded files and chat history are processed in infrastructure the organization controls, which changes what has to be contracted, reviewed and logged. A seat subscription or API typically adds a vendor to that data path. Confirm with your legal and privacy teams which data classes would reach an external service under each option, and price any extra controls those options would need. Build the final comparison as a three-year view with one-time and recurring lines for each option, and request each vendor's quote against the same written usage assumptions.
How to build your own estimate before asking for a quote
This sequence turns a vague budget question into a written set of users, models, hardware, work items and recurring costs that you can compare against per-seat and per-token alternatives over the same horizon. The output is the scope you bring to a scoping call.
Define the user population and peak concurrency
Start by counting the staff who will use the assistant, then estimate how many will chat at the same moment during a busy hour. Concurrency, not headcount, drives GPU size, so ask each business unit for its expected peak and its expected share of heavy users who upload files. Done when you have three numbers: named users, peak concurrent sessions and the share of sessions that use retrieval over uploaded documents. Record the assumptions in a shared sheet so the scoping call can test them.
Choose the models and context sizes
Select the open-weight models the assistant needs, with a general model for drafting and summarizing and, if required, a model for code or other specialist work. For each model, confirm its license terms, the context length your typical prompts and document uploads need, and whether the same model must serve several business units. Larger models and longer contexts require more accelerator memory per request. Done when each model has a named use case, a license review owner and a context size written down.
Size GPU capacity against concurrency settings
Translate the concurrency figures into hardware by testing how the serving layer behaves under your load. With Ollama, check OLLAMA_NUM_PARALLEL (default 1 parallel request per model), OLLAMA_MAX_LOADED_MODELS and OLLAMA_MAX_QUEUE (default 512 queued requests). With vLLM, benchmark throughput and KV cache use on the candidate GPUs and document the tensor parallel setting for each model. Ask the partner to show the benchmark method and the result, and ask the hardware vendor or cloud provider to confirm current pricing for the GPUs you need. Done when you have a GPU count per model, a measured latency target and a list of hardware options, owned or rented.
List integration, security testing and operations work
List every piece of non-model work and name an owner for each: SSO and group mapping, the database and shared storage needed for multiple replicas, the reverse proxy that allowlists user-facing vLLM endpoints, logging and retention settings, security testing of the chat interface and any agent or file-upload features, backup, and user onboarding. Mark each item as one-time or recurring and state who pays for it, whether your staff, an implementation partner or a vendor. Done when every row has an owner, a cost type and a written acceptance check.
Add the recurring items
Recurring costs usually exceed the first build. Add the hours or contract value for platform upgrades, model updates, security patching, on-call support, capacity reviews and log storage, and confirm how long audit logs must be kept under your policies and any applicable rules, such as the six-month minimum for logs under Article 19(1) of the EU AI Act if the system is high-risk. Include electricity, cooling and data center space for owned hardware, or reserved instance costs for cloud GPUs. Done when you have an annual recurring figure and a list of the assumptions behind it.
Compare totals over the same horizon
Price the alternatives over the same number of years: per-seat assistant subscriptions for your user count, and per-token API usage based on the prompt and output volumes you estimated in the first two steps. Put the self-hosted build, the partner-run option and the hosted option in one table with one-time and recurring columns. Note that the per-token model scales with usage while the self-hosted model scales with capacity. Done when every option has a total for the same horizon, and each gap has a named driver you can test.
Bring the scope to a scoping call
Bring the user counts, peak concurrency, model list with license notes, identity provider details, data classes that may be pasted or uploaded, retention requirements, the comparison table and any hardware you already own. DeployOpen issues a quote from a written scope, so the call should confirm which items are in scope, which remain with your team and which need a vendor. Done when the scope document lists deliverables, exclusions and acceptance criteria, and you can match each quote line to a row in your table.
Questions buyers ask about private LLM chat cost and effort
How much does a private LLM chat cost to run?
The cost is the sum of several components, and most of them are not license fees. The main ones are GPU or accelerator capacity (owned hardware or cloud GPU hours), the servers and storage around the chat interface, the database and cache needed for multiple replicas, and the people who install, secure, monitor and upgrade the platform. Open-source inference software and chat interfaces have no license fee to run. Costs recur every month for power, cloud hours, support and staff time, while hardware purchase, initial integration and security testing are largely one-time. The biggest drivers are the models chosen, peak concurrency, context length and how many users upload files. Ask any quote to break these components out separately, and state the assumed user count and concurrency so you can compare it with per-seat subscriptions and per-token APIs on the same basis.
Do Open WebUI, LibreChat or Ollama cost anything to use?
The core software is open source, so you do not pay a license fee to run Ollama or vLLM on your own hardware. Ollama's pricing page states that running models on your own hardware is always unlimited, and its paid plans apply to cloud models. Open WebUI is distributed under the Open WebUI License, which prohibits altering or removing its branding in deployments with more than 50 users in any rolling 30-day period, unless you hold an enterprise license or written permission from the contributors. Enterprise licenses that allow white-labeling are quoted after contacting Open WebUI's sales team with an expected seat count, and pricing is not published. LibreChat is MIT licensed. Confirm the license terms against the version you plan to deploy, because the terms can change between releases.
How long does a private LLM chat rollout take?
The duration depends on the scope you agree in writing, so no fixed timeline applies to every organization. The phases that usually drive it are confirming requirements, procuring and racking GPUs or setting up cloud capacity, connecting SSO and groups, running a pilot with a defined user group, and then phasing in users. Air-gapped or regulated environments add time for change approvals, transfer of model weights and container images, and security testing. Ask for a written plan that lists each phase, the owner of each dependency, and the acceptance criteria before each wave of users. A partner's fixed-scope proposal should state the timeline for the agreed deliverables and the assumptions behind it.
What must our organization provide before the work starts?
Most delays come from decisions the organization has to make, not from the software itself. You will need to name an identity owner who can supply your SSO provider, group structure and SCIM provisioning details, and a data owner who can classify which data may be pasted or uploaded and set chat retention rules. You also need a decision on GPU capacity: whether to buy hardware, rent cloud GPUs, or use existing servers, and which models are approved for which users. Finally, provide network and certificate details, a named security approver, and a business owner who will set acceptable use policy. Make these inputs part of the written scope so that the timeline depends on items both parties have agreed to.
Can our internal team build and run this without a partner?
Yes, an internal team can build and run a private chat assistant if it has the skills for the platform and the surrounding infrastructure. The work requires Docker or Kubernetes experience for the chat interface, PostgreSQL and Redis for multi-replica deployments, SSO and group configuration, reverse proxy and network controls for the model endpoints, and GPU operations for drivers, scheduling and capacity monitoring. The vLLM security documentation, for example, recommends a reverse proxy that allowlists only the endpoints meant for users. Teams should also plan for ongoing upgrades, since these projects release often, and for logging and retention work that compliance reviews will ask about. Assess these skills honestly before committing to an in-house build.
Why would we bring in an implementation partner?
A partner is worth considering when the internal team lacks time or specific skills, or when the rollout must meet compliance evidence requirements on a fixed schedule. DeployOpen plans, implements, integrates, hardens, backs up, upgrades and documents self-hosted open-source platforms in your own cloud accounts, data centers or air-gapped networks. Engagements are typically fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. DeployOpen also runs security testing of applications, APIs and AI systems, and prepares controls and evidence for compliance assessments, although it does not issue audit reports or certifications. Ongoing support and upgrades are available as retainers, and a quote follows from the written scope.
What recurring costs should we expect after launch?
After launch, the recurring costs are GPU or cloud hours, power and cooling for owned hardware, storage for chat history and uploaded files, backups, and staff time for patching, model updates and access reviews. Each new model you add usually increases storage and memory needs, and growth in concurrent users may require more GPUs or additional replicas. Support retainers and periodic security testing are optional but common recurring items. To compare this with per-seat subscriptions, multiply the monthly run cost by your planned user base and then add the one-time items, and compare that total with the subscription or per-token API spend you expect at the same volume. Ask any provider to show the assumptions behind each line.
How DeployOpen scopes and delivers a fixed-scope private LLM chat
DeployOpen plans, implements, integrates, hardens, documents and supports self-hosted open-source chat platforms inside your own cloud accounts, data centers or air-gapped networks. We start from a written scope that separates GPU hardware, inference software, the chat interface, integration work and recurring operations, so the cost comparison with per-seat subscriptions and per-token APIs rests on your numbers.
How we scope
- Written scope lists the user count, concurrency, models, GPU hosting location and data classes the chat will handle.
- Each cost line is named separately: hardware or cloud GPUs, inference software, interface licensing, integration and operations.
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before any work starts.
How we build
- We install the chat interface and inference runtime in your environment using the install methods the vendors document.
- We connect SSO, groups and role-based access to your identity provider and confirm each setting against your own policy.
- We test the inference endpoints, confirm they are not exposed beyond intended users and verify logging and retention settings.
How we hand over
- You receive architecture, configuration and operating documentation covering every recurring cost driver and its owner.
- We document the license terms that apply to your user count and branding, so procurement decisions are based on written facts.
- Evidence for compliance assessments is prepared, though independent auditors and certification bodies issue any assessment outcome.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, priced from the written scope.
- Model and interface updates are reviewed and tested before rollout, so release cadence stays under your control.
- Quotes are issued from the written scope, so you can compare a self-hosted build against a subscription or API with the same assumptions.
