Why private LLM chat in air-gapped networks needs a different build
An isolated network that must give staff a chat assistant has no route to model vendors, package mirrors or update servers, so the build starts from transfer, not download. Every model weight, container image, Python package and documentation set has to be vetted outside the enclosure, carried across the boundary and verified on arrival. The hard parts are artifact transfer, disabling cloud features, offline documentation, update cadence and GPU capacity with no cloud burst. Ollama's FAQ says local runs do not expose prompts to Ollama, and OLLAMA_NO_CLOUD=1 turns off cloud features, but the team still has to confirm every other component makes no outbound call at startup or during use. Offline documentation for the interface and the inference server also has to be staged in advance, and vLLM's --enable-offline-docs option is one item to confirm against the version you run. Plan the update cadence before go-live: vLLM shipped minor releases about every two weeks through 2026, and Ollama ships frequently, so each new release becomes a scanned, signed and tested import rather than a routine upgrade. Capacity is the last constraint, because without cloud burst you size GPUs for peak concurrency, not average use.
For the platform choice, the short answer is Open WebUI or LibreChat on top of Ollama or vLLM. Open WebUI can run entirely offline with Ollama or OpenAI-compatible APIs, and its license limits branding changes above 50 end users in any rolling 30 day period, which matters for an agency or bank that wants its own name on the interface. LibreChat is MIT licensed and is the simpler fit where branding is not a concern. Ollama suits small teams, single-GPU hosts and models that must be loaded quickly, with OLLAMA_NUM_PARALLEL defaulting to one parallel request per model. vLLM suits shared service for many concurrent users, serves OpenAI-compatible endpoints and exposes Prometheus metrics at /metrics, but it needs a reverse proxy that allowlists only user endpoints, because its --api-key option does not cover every route.
Before you choose, confirm three things in writing: the expected peak concurrency, which groups and integrations (SSO, document stores, internal APIs) must work without internet access, and who will carry and verify each release across the gap. Those answers decide the serving engine and the interface, and they set the staffing and GPU budget that a partner should quote.
Which requirements change the build in an isolated network
This table compares the requirements most likely to change how a private chat assistant is built when it runs without internet access. Read each row as a question to settle early with your compliance lead, since applicability depends on your sector, your system classification and how the assistant is used. The main takeaway is that the offline constraint is only the start: software integrity checks, remote access paths and logging duties each add work that is easier to design in than to retrofit.
| Requirement | What it means for the build | Confirm with your compliance lead |
|---|---|---|
| No internet access at runtime | Model weights, container images, Kubernetes control plane images, documentation and updates must all be staged through a controlled transfer path, and cloud-hosted features must be disabled. | Whether the enclave allows any scheduled or one-way transfer, and who approves each import. |
| NERC CIP-005-7 interactive remote access | If the assistant sits near BES Cyber Systems, remote access to its management plane must pass through an Intermediate System with encryption and multi-factor authentication, and vendor sessions must be detectable and disableable. | Whether the assistant or its hosts fall within the electronic security perimeter and which impact rating applies. |
| NERC CIP-013-2 software integrity and authenticity | Every package, model file and container image carried across the boundary needs a verification step, such as checking signatures, hashes and provenance, recorded before it enters the enclave. | Which vendor integrity evidence your procurement process requires for open-source components and model weights. |
| EU AI Act Articles 12 and 19 logging | If the use case is classified as high-risk, the system must technically allow automatic event logging over its lifetime, and logs must be kept for at least six months unless other law sets a different period. Plan log storage inside the enclave from the start. | Whether the staff assistant is high-risk at all, since most internal chat use may not be, and what retention other law requires. |
| Open WebUI branding limit above 50 users | Under the Open WebUI License, clause 4 restricts altering or removing branding unless the deployment has no more than 50 end users in any rolling 30 day period, has written permission, or meets another listed exception. Account for this in the interface design and in user counts. | Your user count in rolling 30 day windows, and whether a license arrangement or a different interface such as LibreChat fits the rollout plan. |
| Offline vLLM endpoint exposure | The vLLM --api-key option does not protect every endpoint, so an allowlisting reverse proxy in front of user-facing routes is the expected pattern. Multi-node traffic must sit on an isolated network. | Whether your security standard accepts a reverse proxy as a control and how its configuration is tested. |
Which open-source chat and model platforms fit an air-gapped network
An air-gapped chat assistant is usually a stack of three or four components: an interface, a local model runtime, a serving layer for shared GPU capacity and sometimes a gateway for routing and logging. The cards below cover the five platforms most often evaluated for that stack and what each one asks the reader to confirm before the network is sealed.
Open WebUI as the chat interface
Open WebUI is the most common browser interface for this stack, and it can run entirely offline when paired with Ollama or an OpenAI-compatible API. The branding clause is the decision point: its license prohibits altering or removing branding except for deployments with no more than 50 end users in any rolling 30 day period, with written permission, or under other listed exceptions. Confirm the license terms against your user count and check that every bundled feature, such as web search or external embedding services, is disabled or pointed at internal endpoints.
LibreChat as an alternative interface
LibreChat is an AI chat interface under the MIT license, so it avoids the branding restriction that applies to Open WebUI. It fits organizations that want a permissive license for the interface and that need multiple model endpoints behind one chat front end. Confirm in your own environment which model providers, plugins and file services it loads at startup, and remove any that reach external hosts. Test offline behavior on the exact release you plan to pin, because interface features change between versions.
Ollama as the local model runtime
Ollama runs open-weight models on local hardware under the MIT license and is the simplest runtime for small teams or single-server pilots. Set OLLAMA_NO_CLOUD=1 to turn off cloud features, and bind the server deliberately, since it listens on 127.0.0.1 port 11434 by default and its local API has no authentication. Model weights must be copied in through a controlled import path, and concurrency defaults such as OLLAMA_NUM_PARALLEL should be measured against real staff load before rollout.
vLLM for shared GPU capacity
vLLM is an Apache-2.0 serving engine with OpenAI-compatible endpoints, and it is the usual choice when many staff share a fixed GPU pool and need batching and tensor parallelism. It has no paid edition, so the cost sits in GPUs, operations and the people who maintain it. Its --api-key option does not cover every endpoint, so place a reverse proxy in front that allowlists only user-facing routes. Confirm the offline documentation option and the image and model paths you will mirror before the network is isolated.
LiteLLM for routing and logging
LiteLLM is an AI gateway that presents more than 100 LLM APIs in OpenAI format, with cost tracking, guardrails, load balancing and logging. It is MIT licensed except for its enterprise directory, so confirm which features your deployment uses fall outside that directory before relying on them. In an air-gapped network it can place one internal endpoint in front of Ollama and vLLM, giving a single point for keys and request records. Verify that its callbacks and telemetry are disabled, and that logs are written to storage inside the enclave.
How to verify model weights and container images before they cross
Every artifact that crosses the air gap should be checked before it is loaded, and the staging process is where that check belongs. For the Kubernetes control plane, run kubeadm config images list on a connected staging host to see the exact image set for your target version, then kubeadm config images pull to fetch it. Mirror those images into a private registry inside the isolated network, and point the cluster at it with kubeadm init --image-repository so no node reaches a public registry during bootstrap. Keep the staging host's pulled image list, the registry digests and the release notes you reviewed together in one transfer record. That record is what an auditor or your own security team will ask for when they trace a running image back to its source.
Model weights need the same discipline. Ollama stores models in a default directory that depends on the operating system, and the OLLAMA_MODELS variable moves that store to a path you control, such as an approved media mount or a dedicated volume on the isolated host. Before import, compare each weight file against the checksum published by the model source and record the manifest that describes its layers. Confirm which licenses travel with each model, because the license terms are part of what you are approving. Where the vendor does not publish checksums for a given artifact, treat that as a gap to resolve with the vendor before the transfer, not after.
This is the control behind software integrity verification, which NERC CIP-013-2 requires in supply chain procurement processes for vendor software and patches. The standard expects you to verify that what you install is authentic and unaltered, and an isolated network with no internet access does not remove that expectation. Write the verification steps into your procedure, assign a named owner for each transfer, and keep hashes and signatures with the records. For the next step on Ollama's model store and its configuration keys, read the Ollama FAQ before you set up the staging host.
Which enclave model changes the transfer and update process
In a fully disconnected enclave, nothing crosses the boundary except through approved removable media or a documented transfer station. Model weights, container images, Helm charts, operating system packages and offline documentation are all prepared outside the enclave, packaged, scanned and hashed, then carried in. Updates follow the same path every time, so the release cadence of fast-moving tools such as Ollama and vLLM becomes a planning input rather than a background task.
- Model weights: Downloaded on a connected staging host, checksummed and imported into the enclave model store, with the storage path set through a variable such as
OLLAMA_MODELSwhen Ollama is used. - Container images: Pulled ahead of time with the registry mirror approach used for Kubernetes control plane images, then pushed to an internal registry inside the enclave.
- Documentation: Offline copies of product and API documentation are shipped with each release so administrators can work without internet access.
- Approvers: Typically the security team, the data owner for the transfer, and the change board. Confirm whether a second person must witness media handling.
- Confirm with your security team: Which media types are permitted, how media is sanitized and logged, and whether the enclave can disable all telemetry and cloud features by configuration.
Ask the security team to write the transfer procedure before the first model is imported, because it defines how every later update is approved.
A controlled one-way import enclave accepts inbound data through a defined path with approvals, but nothing flows back out. Staff in the enclave can receive new model files, images, patches and documentation, while chat content and logs stay inside. The import path is the main control, so it needs an intake point, a scanning step and a recorded approval for each item.
- Intake: A dedicated import host or data diode-style gateway receives items from the connected side and nowhere else.
- Verification: Hashes and signatures are checked against the vendor source before items are released, and package contents are scanned for malware and unexpected dependencies.
- Approvals: Usually a named security approver for each batch, with the platform owner confirming compatibility with the running vLLM or Open WebUI version.
- Outbound restrictions: The enclave must not send prompts, files, chat history or telemetry back to the connected side, so cloud features and update checks need to be disabled and tested.
- Confirm with your security team: Whether the one-way path is an accepted control under your framework, who owns the intake host, and how rejected items are recorded.
The approval record for each import becomes evidence in later assessments, so define its format early.
In this model, a connected staging enclave pulls and tests releases, then produces signed bundles that are promoted into the production enclave. The staging environment has internet access, so it can track upstream releases quickly, but it never runs production user traffic and never holds production chat data. The bundle is the unit of transfer, which keeps the production change small and repeatable.
- Bundle contents: Pinned model versions, image digests, configuration files, offline documentation and a manifest with checksums.
- Testing: Functional and security tests run in staging before the bundle is approved, including checks that the production configuration makes no outbound calls.
- Approvers: The platform owner builds the bundle, the security team approves the content, and the production change board authorizes promotion.
- Separation: Keep staging credentials, registries and model stores separate from production so a compromised staging host cannot write directly into production.
- Confirm with your security team: Whether the staging enclave falls within the same accreditation or assessment boundary, and how bundle signing keys are managed.
Treat the bundle manifest as the change record, because auditors will ask which exact versions reached production and who approved them.
Which components sit inside the enclave and how they connect
A private chat deployment in an air-gapped network is a chain of components that must all work without outside access. Each one has a job in the control set, and each one has a setting that can quietly reopen a path to the internet or to unvetted software.
Private container registry
The registry holds every image the enclave runs, including the chat interface, the inference server, the proxy and the monitoring agents. Images are pulled once on a connected staging network, scanned, signed or hash-checked, then imported across the gap. Inside the enclave, nodes should pull only from this registry, never from public hubs. The configuration risk is a default image reference that still points to a public registry, which fails only when a node is rebuilt. Check every manifest and Helm value for hard-coded registry hosts before cut-over. Kubernetes clusters can be bootstrapped this way, using the image list and private repository flags in kubeadm.
Model store
Model weights are large files that must cross the boundary as a verified bundle, with a manifest listing each file, its version and its hash. The store is the enclave-side location where Ollama or vLLM reads them. It supports the software integrity control in NERC CIP-013-2 when the organization operates in the energy sector, and it supports the same discipline elsewhere. The configuration risk is a runtime that downloads a model on first request, which would reach outside the enclave. Ollama stores models in a configurable directory set by OLLAMA_MODELS, so confirm that path sits on a controlled volume and that the pull step is disabled in operations.
Inference layer on a private address
The inference server is the component that holds the GPUs and answers chat requests. vLLM exposes OpenAI-compatible endpoints and Prometheus metrics at /metrics, which feeds the monitoring node. Ollama binds to 127.0.0.1 on port 11434 by default and offers no authentication on the local API, so it must sit on a private interface reachable only from the proxy. The configuration risk is an inference port listening on all interfaces. For vLLM, the --api-key option does not cover every route, so do not rely on it alone. Confirm the bind address, disable cloud features in Ollama with OLLAMA_NO_CLOUD=1, and check that no outbound route exists from the GPU hosts.
Allowlisting reverse proxy
The reverse proxy is the only path from staff workstations to the inference layer. It should pass the user-facing routes, such as chat completions and model listing, and block administrative and diagnostic routes such as metrics, pause, resume and abort. The vLLM documentation recommends this pattern because its API key does not protect every endpoint. The proxy also terminates TLS with enclave-issued certificates and can enforce the identity checks that the chat interface alone may miss. The configuration risk is a catch-all location block that forwards everything to the backend. Test it by calling each blocked route from a user subnet and recording the denied response.
Chat interface on the identity provider
Open WebUI or LibreChat runs as the staff-facing chat front end, and it should authenticate users against the organization's identity provider inside the enclave, such as an on-premises directory or Keycloak. Group membership should decide who can reach which models and who may upload files. Open WebUI can run entirely offline with Ollama or OpenAI-compatible APIs, so the front end itself needs no internet access. The configuration risk is a local admin account left enabled with a default password. Disable local sign-up, confirm that every session is tied to a directory identity, and check that the interface does not fetch external assets or update checks at start-up.
Monitoring node with offline updates
A monitoring node collects logs, alerts and integrity findings from the proxy, the inference hosts and the chat interface. Wazuh publishes an offline installation guide and can use an offline vulnerability repository in place of its online threat intelligence feed. Security Onion supports airgap installs, carries its own packages and rules, and updates from newer ISO media through its soup tool. Logs from this node support the record-keeping duties in Article 12 of the EU AI Act, where that regulation applies. The configuration risk is an update path that quietly depends on a feed the enclave cannot reach, leaving detections stale without a visible error. Track rule and vulnerability data age as a monitored metric.
How an air-gapped private LLM chat rollout runs in phases
The sequence moves a private chat assistant from scoping to a supported, updated service inside an isolated network. Each phase closes with a document or test result the next phase depends on, so the rollout can be checked by the security team at every gate.
Scope the use case and classify the data
Define who uses the assistant, which data staff may paste or upload, and which models and interfaces are in scope. Your compliance lead and DeployOpen review the use case together, including whether the organization deploys the system only or builds it in-house, which affects its position under the EU AI Act. The output is a signed scope document with a data classification table, a user and concurrency estimate to confirm with your team, and a list of the open-source components and their licenses. Branding limits in the Open WebUI License apply above 50 end users in a rolling 30 day period, so confirm them here.
Build the staging bundle and manifest
On a connected staging network, collect model weights, container images, Helm charts, operating system packages and the offline documentation the platform needs. Record every artifact with its version, source URL and cryptographic hash in a manifest. Verify each package against its publisher's signature where one exists, which supports the supply chain integrity controls that NERC CIP-013-2 describes for utilities and that many other regulators expect. The output is a versioned bundle and a manifest that the enclave team can check line by line.
Transfer and verify the bundle into the enclave
Move the bundle across the boundary under your approved media transfer procedure, then recompute the hashes inside the enclave and compare them with the manifest. Load container images into a private registry and point the cluster or host at it, since Kubernetes can be built from pre-pulled images and a private registry. Any mismatch stops the transfer and returns the bundle to staging. The output is a transfer record signed by both sides and a registry that contains only manifest-listed images.
Install with cloud features off and offline docs enabled
Deploy the model server, such as Ollama or vLLM, and the chat interface, Open WebUI or LibreChat, with no outbound route to the internet. Set OLLAMA_NO_CLOUD=1 so cloud model features stay disabled, keep Ollama bound to its local address or behind a proxy, and load models from the path set by OLLAMA_MODELS. For vLLM, enable the offline documentation option, place an authenticating reverse proxy in front of user-facing endpoints, and confirm that the operational endpoints are not reachable from user networks. The output is a configuration baseline and a test report showing that outbound connections are blocked and the chat works end to end.
Pilot with capacity tests against metrics and concurrency settings
Run a limited pilot group and load test the stack with realistic prompts and file uploads. Tune OLLAMA_NUM_PARALLEL, OLLAMA_MAX_LOADED_MODELS and OLLAMA_MAX_QUEUE for Ollama, or the equivalent serving settings for vLLM, and watch request counts, token throughput, latency and KV cache use on the Prometheus-format endpoint at /metrics. Confirm the sizing against the GPUs you have, since an air-gapped site cannot burst to cloud capacity. The output is a capacity report with measured limits, the concurrency settings chosen, and a go or no-go decision for wider rollout.
Set the recurring update cycle
Agree a fixed cadence for reviewing and carrying new releases across the gap, because Ollama and vLLM publish releases frequently. Each cycle repeats the staging, manifest, transfer and regression steps, and rolls back to the prior bundle if a test fails. Ask your partner and your security team to confirm which releases are security fixes and which are feature updates, and set the review rules accordingly. The output is a documented update runbook, a release log tied to each manifest, and a retained copy of the last known good bundle.
What auditors and regulators ask for from the enclave
This table lists the evidence items an assessor is likely to request from an air-gapped chat deployment, what each item shows and which component produces it. Read it as a checklist to agree with your auditor before build starts, since DeployOpen prepares this evidence but independent auditors, certification bodies and regulators draw the conclusions.
| Evidence item | What it shows | Component that produces it |
|---|---|---|
| Transfer manifest and verification records | Every model weight, container image and package that crossed the gap, with checksums, signatures and who approved each transfer | Transfer workstation scripts and a signed manifest kept on the high side |
| Approved model and image versions | The exact model tags, inference engine and chat interface versions allowed inside the enclave, with license notes | Internal registry catalog and a version register maintained by the platform owner |
| Change records for each update bundle | What changed in each bundle, why, who tested it and when it was promoted to production | Ticketing system linked to the bundle manifest and test results |
| Chat and inference logs | Who asked what, which model answered and when, retained for at least six months where Article 19(1) of the EU AI Act applies | Chat interface database, inference server request logs and a log store inside the enclave |
| Access and allowlist configuration | Which endpoints users can reach, which admin paths are blocked and which groups hold chat or admin roles | Reverse proxy allowlist, SSO group mappings and chat interface role settings |
| Monitoring alerts and responses | Alerts for failed transfers, unexpected outbound connections, endpoint errors and capacity saturation, plus how staff responded | Metrics stack scraping inference metrics, network monitoring and the alert log |
| Network boundary tests | Evidence that no outbound route exists and that cloud features are disabled, such as server settings and firewall test results | Firewall rule export, egress test reports and configuration files for the chat and inference servers |
Which questions to ask an air-gapped chat implementation partner
Start with the transfer path, because every model weight, container image and package that crosses the gap is a supply chain decision. Ask the partner how they build the transfer bundle, which checksums or signatures they verify before import, and who signs off on each bundle. Ask them to name the private registry that will hold container images and the model store where Ollama keeps weights (set with the OLLAMA_MODELS variable) or where vLLM loads them. Then ask what the enclave update cadence looks like in practice: how often new Ollama and vLLM releases are reviewed, how long a bundle takes to move from the outside network into the enclave, and what happens when a security fix lands between cycles. Ask how they meet the software integrity expectations in NERC CIP-013-2 if that standard applies to you, and which parts of the process they document for your assessor. Ask for the written procedure, not a verbal summary.
Ask how the partner confirms that nothing in the stack calls out. Ollama binds to 127.0.0.1 on port 11434 by default, and its cloud features can be turned off with OLLAMA_NO_CLOUD=1 or in the server configuration file, so ask who sets those values, where they are stored and how drift is detected. For vLLM, ask how telemetry is disabled, which environment variables they set, and how they handle the Open WebUI settings that could fetch external resources. Ask whether they test the boundary with a deny-by-default egress policy and a packet capture during a full chat session, including file upload and model load. Request the test scripts and the result from a recent run, and ask which findings they expect to see and how they would report a failure. Also ask which of the tools they propose need a paid edition for air-gapped operation. Vendor pages show that air-gapped support is often restricted to enterprise plans, so get that answer in writing for each component.
What do buyers ask about air-gapped private LLM chat cost and effort?
What drives the cost of an air-gapped private LLM chat build?
Most of the cost sits in GPU capacity bought up front, because an air-gapped enclave cannot burst to cloud accelerators when demand rises. The main components are the GPU servers and storage, a staging environment outside the enclave where models and container images are downloaded and checked, the implementation and integration work, and recurring updates. Ask any quote to separate one-time hardware and setup from the ongoing work of moving each model and image release across the boundary and testing it. Sizing for peak concurrency, not average use, is the single largest driver of hardware spend.
Do the open-source model runtimes cost money to run?
The core runtimes carry no license fee for local use. Ollama's pricing page states that running models on your own hardware is always unlimited, and its paid plans cover cloud-hosted models, which an air-gapped build does not use. vLLM is Apache-2.0 licensed and has no paid edition. Your spending therefore goes to hardware, the chat interface if you choose a commercial tier, and the people who operate the stack. Confirm the license of every component you plan to use, including the chat interface, because some interfaces restrict branding changes above a user threshold.
Is air-gapped support a paid feature in other tools?
Often, yes. Several vendors reserve air-gapped deployment for paid editions: Mattermost lists it under Enterprise Advanced, Rocket.Chat excludes it from its free Starter plan, and Metabase offers it only on its Enterprise plan. Element Server Suite separates a free AGPL Community edition from a Pro edition and a Sovereign edition for air-gapped use, and Plane offers a separate Airgapped edition that is proprietary. Before you choose a supporting tool for the enclave, check the vendor's current pricing page and confirm in writing which edition includes offline operation.
How long does an air-gapped build take, and what changes the timeline?
The duration depends on scope, not on a standard schedule. The main variables are how many models and use cases the first release covers, whether the Kubernetes or Docker platform already exists in the enclave, how many identity and document integrations are needed, and how long your security team takes to approve the transfer process and the test evidence. Hardware lead times can dominate the early phase. DeployOpen fixes scope, deliverables and timelines in writing before work starts, so ask for a dated plan built from your environment rather than a generic estimate.
What must our organization supply before implementation starts?
The buyer typically supplies the enclave network design, the hardware or the approved purchasing path for GPUs, an internal container registry and package mirror, the approved list of models and their licenses, identity provider details for SSO, and named data owners for any documents the assistant may read. You also need a written cross-domain transfer procedure and an accountable owner for it. Missing items in this list are a common cause of delay, so confirm each one at scoping.
Can our own team operate and update the air-gapped assistant?
Yes, if your team has Linux, Kubernetes or container operations skills and the staffing to run the transfer workflow on a regular cycle. Ask whether your team can run model updates, restore from backup, rotate keys and read vLLM metrics at /metrics without outside help. An implementation partner can document these runbooks and train the operators, but recurring operation is a staffing decision. Compare the cost of an internal team with a retainer for upgrades and support, and confirm which of the two you are buying.
How often do updates arrive, and how do we bring them across the gap?
Release cadence is fast. vLLM has shipped new minor releases about every two weeks in 2026, and Ollama publishes frequent releases, including pre-releases. Each update needs to be downloaded outside the enclave, checked for integrity and authenticity, scanned, tested in staging and then transferred in. Plan a fixed update window rather than reacting to each release, and decide which releases you will skip. Ask your partner how the transfer, verification and rollback steps are recorded for audit.
Can the assistant burst to cloud GPUs when demand rises?
No. Inside a truly air-gapped enclave, there is no path to a cloud GPU pool, so capacity must be sized for expected peak concurrency from day one. The practical options are to buy more on-premises accelerators, queue requests with explicit limits, or serve smaller models to more users. Ollama's concurrency settings, such as OLLAMA_NUM_PARALLEL and OLLAMA_MAX_QUEUE, and vLLM's scaling options help define those limits. Ask any partner to show the measured load test that supports the hardware count in their quote.
How DeployOpen scopes and supports an air-gapped chat build
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own air-gapped networks. For a private chat build, the work starts with a written scope that fixes the models, interfaces, transfer process and acceptance tests before any media crosses the gap.
How we scope
- Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
- We map which models, chat interface, inference server and container images the enclave needs, and which cloud features must be disabled.
- We agree the transfer workflow for model weights and images, including how each package is checked for integrity before it enters the enclave.
How we build
- We prepare the Kubernetes or host platform so it installs from a private registry without internet access.
- We configure Open WebUI or LibreChat with Ollama or vLLM, locking endpoints to users and placing inference behind a reverse proxy that allowlists user-facing paths.
- We run security testing against the chat application, its APIs and the inference endpoints, then fix findings before acceptance.
How we hand over
- We deliver runbooks for the offline update cycle, model onboarding, backups and restore of chat history and configuration.
- We document the GPU sizing assumptions, concurrency settings and monitoring so your team can check capacity against real usage.
- We prepare the compliance evidence set, such as configuration baselines, access records and logging settings, for your own auditors to review.
How we support
- Ongoing support, upgrades and continuous testing are available as retainers, scoped in writing.
- Upgrade work covers reviewing each release of the chat interface, inference server and models before it is carried across the gap.
- Support terms, response expectations and the boundaries of the retainer are confirmed in the written agreement before work begins.
