Private LLM chat for the enterprise

GPU type and count usually sets the largest hardware line item, because the GPUs must hold model weights and serve answers at acceptable speed. Cost rises with memory per card, the number of cards needed to host the chosen model, and whether you buy or rent. Owned hardware is a one-time purchase plus power, cooling, rack space and maintenance, which recur. Rented cloud GPUs shift most of the cost into a recurring hourly or monthly charge. Ask for a written quote that lists each GPU model, card count, memory per card, and whether the price assumes purchase, reserved capacity or on-demand use. Confirm lead times and supply with your hardware vendor, since these change and affect the rollout date.

Yes. Larger models need more GPU memory and more compute per answer, so they typically require more or larger cards. Quantization stores model weights at lower numeric precision, which reduces memory needs and can lower the number of GPUs required, usually at some cost to output quality. The trade-off must be tested on your own tasks before you size hardware. Ask the implementation partner to document which models were evaluated, the quantization format used, measured response quality on representative prompts, and the hardware each option needs. Model licenses also differ, so confirm the terms for each model you plan to serve.

Concurrency, meaning how many users generate answers at the same moment, determines how much serving capacity you need, while average usage matters less. Teams that all work during the same hours create peaks that can be several times the average load. You can add capacity by adding GPUs or replicas, or accept longer queues at peak. Engines such as Ollama and vLLM handle parallel requests differently, so load testing with your expected user count is the only reliable way to size the cluster. Ask the partner to show the load test results, the target response time under peak, and the number of GPUs each scenario requires.

High availability adds redundancy, which means extra GPU capacity, a second node or cluster, load balancing, and failover for the chat front end and database. These costs recur because idle standby capacity still draws power, rent or support contracts. The decision depends on whether a multi-hour outage stops critical work or only slows convenience use. Many organizations start with a single serving node and a tested restore procedure, then add redundancy once usage justifies it. Ask for the availability design as a separate line in the quote, so you can decide which tier to fund.

Storage is usually a small share of the total, but it grows steadily and must be backed up and retained according to policy. Costs come from the database holding conversation history, the file store for uploaded documents, vector indexes if you use retrieval, and backup copies in separate locations. Retention settings decide how fast this grows and how long you pay for it. Logs kept for record-keeping may also need longer retention than chat history. Ask the partner to estimate storage growth from your expected usage, state the retention periods per data type, and price backups and restore testing separately.

Staff time covers operating the platform, applying updates, managing user access, watching capacity, handling incidents and reviewing model changes. Much of this effort recurs every month, and it is often underestimated because it is shared across infrastructure, security and application roles. Upgrades of the chat interface, serving engine and models each need testing before deployment. Decide early whether your own team will operate the system or whether a partner will provide ongoing support under a retainer. Ask the partner to list the operational tasks by role and frequency, and to name who is responsible for each.

One-time costs include hardware purchase, network and storage setup, initial configuration, single sign-on integration, security hardening and the first rollout to teams. Recurring costs include power, cooling, hosting or cloud GPU charges, licenses where a product or model requires them, support contracts, backups, model updates and staff time. Open-source software removes most license fees for the core platform, but licensing terms still matter for branding, multi-tenant use and commercial models. Ask for the quote in two groups, one-time and recurring, with the assumptions behind each figure, including the usage level it was sized for.

A useful quote ties every price to a stated scope, so you can compare proposals and see what changes if requirements change. It should list the GPU hardware or rental assumptions, the models and serving engine, the number of users and expected peak concurrency, availability targets, storage and retention assumptions, integration and SSO work, documentation, training, and ongoing support. It should also separate one-time work from recurring charges and state what is excluded. DeployOpen does not publish a price list. Its fixed-scope engagements are agreed in writing before work starts, so ask any provider for the same breakdown before comparing totals.

Yes, a capable platform team can deploy Open WebUI or LibreChat with Ollama or vLLM, but the work is more than installing containers. The team needs to own the model serving layer, identity integration, backups, upgrades and monitoring, and to decide who answers when the assistant misbehaves. Most of the risk sits in the parts that are not installation: access control mapped to your directory groups, retention settings that match your policy, and a documented upgrade path. Before starting, list the people who will carry each of those tasks after launch, and check whether they have the time to do it alongside their current work.

The cost has three main parts, and only some of them recur. Hardware or cloud GPU capacity for model serving is the largest and recurs for as long as the service runs. Hosting for the chat interface, database, storage and backups is smaller and also recurs. Implementation labor, including SSO setup, testing and rollout, is one-time, while support and upgrades are ongoing. GPU size depends on the models you choose, how many users chat at the same time and the context length they use. No supplied figures fix these numbers, so ask any provider to show the GPU type, count, hours and storage assumptions behind its estimate, and to separate one-time and recurring lines.

The license of the chat interface decides whether you can brand it, extend it and how many people can use it without extra terms. Open WebUI is distributed under the Open WebUI License, which restricts altering or removing its branding unless a listed exception applies, such as deployments with no more than 50 end users in any rolling 30 day period or written permission. LibreChat is MIT licensed. Ollama is MIT and vLLM is Apache-2.0, but the model weights you run carry their own terms, which must be checked separately. Read the license text for your exact version and confirm with your legal team before a company-wide rollout.

Setup time depends mainly on the number of integrations and on how much of the environment already exists. A pilot on a single server with local accounts can be prepared quickly, but a production service with SSO, group-based access, approved model choices, logging and backups usually takes longer, and the GPU procurement lead time can dominate the schedule. Rollout to teams adds time for training, feedback and policy sign-off. Ask any partner to split the timeline into phases with named exit criteria, and to state which dependencies, such as hardware delivery or identity changes, sit outside its control.

DeployOpen plans, implements, integrates, hardens, backs up, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For a private LLM chat, a fixed-scope engagement would typically set out the model serving setup, the chat interface, SSO and access control, retention and backup configuration, testing and handover documentation. Scope, deliverables and timelines are agreed in writing before work starts. Ongoing support, upgrades and continuous testing are available as retainers, so the post-launch work can be priced separately from the build.

A useful quote lists every cost line and the assumptions behind it, so you can compare options fairly. Ask for:

• The written scope, deliverables and acceptance criteria
• The hardware or cloud GPU sizing, with model names, user counts and concurrency assumed
• Which costs are one-time and which recur, including support and upgrades
• How SSO, retention and backup requirements are covered
• Licensing checks for the chat interface and the model weights
• The phases, dependencies and what the client team must provide

DeployOpen does not publish a price list, and a quote follows from the written scope. Ask for the same breakdown from every bidder so the comparison is like for like.

Retention settings change storage cost and the compliance evidence you hold. If the assistant could fall within the scope of high-risk AI rules, Article 12(1) of the EU AI Act requires that automatic recording of events be technically possible over the system's lifetime, and Article 19(1) requires providers to keep those logs for at least six months unless other law provides otherwise. Deciding what to log, how long to keep conversations and who can read them takes legal and security input before the build starts. Confirm with counsel whether the rules apply to your use case, because the answer shapes the storage and access work.