Self-hosted Glean alternative for air-gapped networks

Why an air-gapped network needs a different stack than Glean Customer Hosted

An air-gapped network needs an assistant that works with no outbound path at all, and Glean Customer Hosted does not meet that test by default. The model runs in the customer's own GCP or AWS account, but Glean's documentation says Glean remains responsible for infrastructure and application management, including upgrades, patching and monitoring. That management relationship depends on connectivity and on Glean's operational access to the deployment, which an enclave where no traffic may leave cannot provide. Whether any Customer Hosted component can run with all external paths closed, and how Glean's update and telemetry flows behave in that state, are questions to put to Glean in writing before any architecture decision.

The short answer for an isolated network is a different stack: Onyx deployed as a self-contained service set, with Ollama or vLLM serving the models inside the enclave. Onyx documents an air-gappable deployment in which the document index, database and processing run within its own services, deployable with Docker, Kubernetes, Helm or Terraform, and it connects to self-hosted models through Ollama, LiteLLM or vLLM. Ollama's local API does not require authentication and, by its documentation, does not see prompts when run locally, while vLLM exposes an OpenAI-compatible server. Your team, or a partner, then owns the images, the update path and the model artifacts.

This page covers the isolation constraint and the stack that follows from it. It does not establish that any product is certified for classified or regulated enclaves, and it does not replace your security authority's accreditation process. Confirm with Glean and with each open-source project which components phone home, how offline license checks behave, and what must be transferred in by hand. Those answers decide how much of the stack can be brought in, verified and maintained without a connection to the internet.

What an air-gapped deployment requires from the platform

An air-gapped Glean alternative has to work with no path to outside services at runtime, and that rules out any design where the assistant, index or model depends on a hosted API. The cards below set out the requirements that follow from this, with the supported facts that address each one and the points to confirm in your own environment.

No runtime dependency on external services

Every component the assistant calls during a query must resolve inside the enclave. Onyx documents an air-gappable deployment in which the document index, database and processing run within a self-contained set of services. Confirm the full image list, any telemetry or license checks, and whether cloud features in your chosen components can be disabled. Ollama can be set to OLLAMA_NO_CLOUD=1 so that cloud features are off.

Models served inside the enclave

Inference has to run on hardware you control, with no calls to proprietary model APIs. Onyx works with self-hosted models through Ollama, LiteLLM or vLLM. Ollama's local API at localhost:11434 needs no authentication, so the bind address and network path must be locked down. For vLLM, the security docs say that inter-node traffic is insecure by default and must sit on an isolated network.

Offline indexing from internal sources

Connectors must pull from systems inside the network, such as SharePoint, Confluence, Jira, Google Drive or Slack, and index content without reaching the internet. The Onyx connectors directory lists the sources it supports, but whether each connector works against your self-hosted or on-premises version of a source is something to test. Ask Glean how its Customer Hosted model handles connector traffic, since its deployment docs do not establish offline operation.

Controlled and verified updates

In an isolated network, updates arrive by a transfer process your team approves, not by pulling from a registry at runtime. Plan a mirror of container images, Helm charts and model files, plus checksums for each artifact. Onyx publishes frequent releases and a Helm chart, and Ollama and vLLM ship new versions regularly, so the update path and rollback steps need a written procedure before go-live.

Permission-aware answers

An assistant that searches a whole corpus has to respect the access rules of each source. Onyx syncs permissions from connected applications, so answers only draw on documents the user can already see in the source system. Test this with a set of restricted accounts before rollout, and confirm that group changes in the source propagate on the schedule your policy requires.

Audit and access logging

Usage and query history must stay in your own systems so that investigators can read them. Onyx lists query history auditing and usage analytics among its enterprise features, and SSO through OIDC or SAML with SCIM provisioning ties access to your identity provider. Confirm which features need the Enterprise Edition, and decide where logs from the model server and vector store are retained.

Which open-source options fit an isolated network

This table compares the open-source interfaces and model runtimes that can run without internet access, with the licensing, branding and feature gaps that matter for an air-gapped assistant. Read each row against your own connector list, user count and access control requirements, then confirm the open items with the vendor or in a test environment before you choose.

Option	Role on an isolated network	Licensing	Trade-offs and gaps to confirm
Onyx Standard	Document index, search and chat over company content, deployable with Docker, Kubernetes, Helm or Terraform as an air-gappable stack	MIT for Community Edition; ee directories under the Onyx Enterprise License	Runs Postgres, Vespa or OpenSearch, Redis and model servers, so more services to operate offline; confirm image delivery and update packaging for your network
Onyx Lite	Lighter chat and agents deployment without a vector database or background workers	MIT for Community Edition; ee directories under the Onyx Enterprise License	No indexing workers means no background sync, so content indexed offline is not refreshed automatically; confirm how you will load new documents
Onyx Enterprise Edition	Self-hosted premium tier with features such as RBAC, SCIM provisioning and usage analytics	Onyx Enterprise License for ee code	Role-based access control is an enterprise feature, so plan for licensing if you need it; confirm terms with Onyx before scoping
LibreChat	Multi-user chat interface that can point at Ollama or other local endpoints	MIT	Strong on providers and MCP agents, but connector coverage for company content is not its core; pair with a separate search layer if you need it
AnythingLLM	Desktop or Docker assistant over ingested documents with local models and multiple users	MIT	Suited to smaller document sets and team use; confirm how permissions, audit trails and scale behave against your access model
Open WebUI	Self-hosted interface that runs fully offline with Ollama or OpenAI-compatible APIs	Open WebUI License, BSD-3-Clause style with a branding clause	Removing or altering branding needs 50 or fewer users in a rolling 30-day period, permission, or an enterprise license; review this before rollout
Ollama	Local model runtime with an API on port 11434 that needs no authentication when local	MIT for the runtime	Default bind is 127.0.0.1; if you expose it, put it behind an authenticated proxy and set OLLAMA_NO_CLOUD=1 to keep cloud features off
vLLM	Production model serving with OpenAI-compatible endpoints and Prometheus metrics, suited to GPU clusters	Apache-2.0	Multi-node traffic is insecure by default and needs an isolated network; --api-key does not cover every endpoint, so use an allowlisting reverse proxy

Ollama or vLLM for inference without internet access

Ollama suits a small group that needs a local model on a CPU or a single workstation. It runs as a local runtime, pulls models into a directory you choose, and needs no outbound connection once the model files are in place. Its default local API is convenient for a single machine but was not designed to be exposed to a network, so the surrounding controls matter more than the runtime itself.

Local API without authentication: The API on port 11434 accepts requests with no credentials. Ollama binds to 127.0.0.1 by default, so any wider exposure through OLLAMA_HOST should go behind a reverse proxy such as Nginx that enforces authentication.
Cloud features off: Set OLLAMA_NO_CLOUD=1 or the equivalent server configuration entry so the server never attempts direct cloud inference.
Concurrency limits: OLLAMA_NUM_PARALLEL defaults to 1 request per model, OLLAMA_MAX_LOADED_MODELS and OLLAMA_MAX_QUEUE cap loaded models and queued requests. Plan these for the number of users you expect.
Model storage: Use OLLAMA_MODELS to move model files to a managed volume, and copy them through your approved media transfer process.
Costs: Running models on your own hardware carries no per-token fee, but hardware, patching and staff time recur.

Confirm with your security team whether a single-user workstation or a shared server is the right target before you open any port.

vLLM is an inference and serving engine built for throughput on accelerators. Its vllm serve command exposes OpenAI-compatible endpoints, so an assistant such as Onyx can call it like a hosted API. It is the more likely choice when many users share a model, but it needs more planning for access control and hardware.

OpenAI-compatible endpoints: The server offers /v1/completions, /v1/chat/completions, /v1/responses and /v1/embeddings, among others, so applications written for the OpenAI API can point at it.
Hardware support: vLLM runs on NVIDIA CUDA, AMD ROCm and Intel XPU GPUs, and on several CPU architectures. Confirm the exact driver and accelerator matrix for your chosen version in the installation docs before you buy hardware.
The `--api-key` limit: The key protects /v1, /v2, /inference and /cohere, but not /invocations, /pause, /resume, /abort_requests, /metrics or /health. The project warns against relying on the key alone.
Reverse proxy allowlist: Place a proxy in front of the server that forwards only the endpoints users need, and keep administrative and metrics paths on the internal network.
Multi-node communication: Communication between nodes is insecure by default, without authorization or encryption. Isolate the cluster network and treat it as a trusted segment.

Ask your platform team to confirm the proxy allowlist and the isolated node network in writing before any model is loaded.

Large models often need several GPUs or several machines. vLLM splits a model with tensor parallelism inside a node and pipeline parallelism across nodes, and it uses Ray or its own multiprocessing backend for multi-node runs. This is where the default trust assumptions of the cluster matter most, because the nodes exchange model state with each other.

Parallelism choice: Tensor parallelism works within one node, while pipeline parallelism spans nodes. Each added node increases network traffic and the number of components to patch.
Node traffic: Node-to-node traffic has no built-in authorization or encryption, so it must stay on a segregated network with strict firewall rules.
Backend choice: Ray and the built-in multiprocessing backend have different operational footprints. Document which one your team will support.
Kubernetes options: The vLLM docs list native manifests, Helm, the production-stack, KServe and KubeRay, among others. Each option brings its own cluster hardening work.
Metrics exposure: The /metrics endpoint exposes request counts, token throughput, latency and KV cache use. Restrict it to your monitoring network.

Request a written network diagram from your implementation partner that shows which ports cross each node boundary before you deploy.

How to deploy Onyx and its models inside the enclave

This sequence moves Onyx, its container and chart artifacts, and model weights into an isolated network, then ends with a deployment that answers from internal sources and is checked for outbound connections. Each step produces a record your security and platform teams can review.

Pin release versions and verify artifacts
Platform engineering selects the Onyx release and matching Helm chart version from the project's GitHub releases page, then records the exact tags for the Onyx images, the chart and any inference server images. Verify each artifact against its published checksum or signature where the project provides one, and confirm the version with your own change process before anything crosses into the enclave. Done when a signed-off manifest lists every image digest, chart version and model file hash that will be imported.
Stage images, charts and model weights
Using your approved transfer path, such as a cross-domain transfer process, removable media under your media policy, or an internal mirror, move the container images, Helm chart, deployment files and model weights into the enclave. Load the images into an internal registry and store model files on a volume the inference server can read. Done when every item on the manifest exists inside the enclave with a matching hash, and the registry serves the pinned tags.
Deploy Onyx Standard or Lite
Choose Onyx Standard if you need vector and keyword indexing with background sync workers, or Onyx Lite if you only need chat and agents without a vector database or background workers. Deploy with Docker, Kubernetes with Helm, or Terraform, pointing the images at the internal registry and keeping Postgres, the index engine, Redis, object storage and model servers inside the enclave. Done when all pods or containers report healthy and the chat interface loads from an internal address only.
Connect Onyx to Ollama or vLLM
Configure Onyx to call the model server inside the enclave, either Ollama, which exposes a REST API on port 11434, or vLLM, which serves OpenAI-compatible endpoints through vllm serve. For Ollama, keep the bind address on the internal interface and confirm that its local API has no authentication, so restrict access with network policy. For vLLM, place a reverse proxy in front that allowlists only the endpoints Onyx needs, because the --api-key option does not cover every route. Done when a test question returns an answer generated by the internal model with no external call logged.
Configure SSO through OIDC or SAML
Work with your identity team to register Onyx with the internal identity provider using OIDC or SAML, and enable SCIM user provisioning if your policy requires automated joiner and leaver handling. Confirm the mapping of groups to Onyx roles and check which features need the Enterprise Edition, such as role-based access control. Done when a test account signs in through the internal identity provider, and a disabled account is refused access.
Set up connectors to internal sources
Create connectors for the internal sources you approved, such as SharePoint, Confluence, Jira or file shares, using service accounts with read-only scopes. Enable permission sync so answers draw only on documents each user can already see in the source system, and test that a restricted document does not appear for an unauthorized account. Done when each connector shows a successful sync and a permission test passes for at least one restricted document per source.
Run initial indexing offline
Start the first full index from the internal sources and monitor the sync workers and index service until the job completes. Record the document counts, failed items and any connector errors, then review them with the source owners. Schedule later incremental syncs to run only inside the enclave, and note that any new model or Onyx release must go through the same import and verification path. Done when the index reports the expected document set and a query on a known internal topic returns the correct source.
Validate that no egress occurs
Security engineers test the enclave boundary by blocking outbound traffic at the firewall and then exercising the full workflow, including login, indexing, chat and model inference. Review DNS queries, proxy and firewall logs for any attempted connection to external hosts, and confirm that Ollama cloud features are disabled with OLLAMA_NO_CLOUD=1 or the server configuration file. Done when the test log shows no outbound connection attempts and the evidence package is filed for your assessment team.

How indexing and updates work when the network is offline

Connectors only work if they can reach the systems they index, so the source applications themselves must sit inside the isolated network. Onyx pulls content from SharePoint, Confluence, Jira and similar sources through its connectors, and it syncs permissions from those sources as well. Before go-live, the team should map each connector to a source host on the enclave, confirm service accounts and API access inside that boundary, and check that the sync schedule does not depend on any external endpoint. Any source that lives outside the enclave will not be indexed, and the gap should be recorded as a scope decision rather than discovered by users.

Updates reach the enclave by transfer, not by pulling from the internet. The team should stage each Onyx container image, the Helm chart, Ollama or vLLM binaries and model weight files on a controlled import path, such as approved removable media or a one-way transfer diode, and verify checksums or signatures before anything is deployed. Onyx publishes releases on GitHub and also a separate Helm chart version, so the two must be tracked as distinct artifacts. Ollama and vLLM also ship minor releases on a frequent cadence, and the vLLM project's own releases list shows new minor versions roughly every two weeks. Someone on the team must own a regular assessment of which releases to import, with a written record of what was tested in a staging copy of the enclave.

Ask Onyx in writing how background sync, permission refresh and index updates behave when the deployment has no outbound connectivity, including whether any license check, telemetry or model download is attempted at startup. Confirm how a permission change in a source system, such as a removed group membership, propagates to the index when sync runs on the isolated network, and how long a stale permission can persist. Also confirm how a failed sync is reported and whether a re-index can be triggered manually. Document the answers and test them with a revoked user before production, because the access controls on answers depend on them.

Evidence an auditor or regulator will ask for

An air-gapped deployment is credible only when its isolation, configuration and change history can be shown on paper and in logs. The cards below cover the artifacts most reviewers request, and DeployOpen prepares them, but auditors and regulators decide whether the evidence is sufficient for your obligations.

Network diagrams showing no egress

Diagrams should show every Onyx service, the database, the index, Redis, model servers and the object store inside the isolated zone, with each allowed flow listed. Mark the boundary firewall rules and any one-way transfer point used for imported artifacts. Ask your team to test the claim by attempting outbound connections from each host and keeping the results, because a diagram alone does not prove that no egress path exists.

Configuration exports for Onyx and model servers

Export the effective configuration for Onyx, the model server and any vLLM or Ollama instance, and store each export with a date and an owner. Confirm that the Ollama setting OLLAMA_NO_CLOUD=1 is applied and that the Ollama bind address is what you documented. Also record vLLM's --api-key setting and the reverse proxy allowlist that exposes only user-facing endpoints.

Access and query history logs

Reviewers usually ask who signed in, which sources a user could search and what queries ran. Onyx lists query history auditing and usage analytics among its enterprise features, so confirm which of these your edition provides and where the logs are stored. Set a retention period for logs, protect them from edits and confirm that they stay inside the isolated network.

Update and change records for imported artifacts

Because the network cannot fetch updates, every container image, Helm chart, model file and package is imported through a controlled process. Keep a record of each artifact's source, checksum, approval, import date and the person who moved it across the boundary. Also log every Onyx release you adopt, such as the version you tested before import, so the change history matches what runs in production.

Permission sync tests

Onyx connects to 50+ applications and syncs source permissions, so answers should only draw on documents the user can already see. Keep test cases that use accounts with different access rights, run queries that should return nothing for restricted users, and retain the results. Repeat the tests after each connector sync and after each import, since the offline sync path is where permissions most often drift.

Model and version inventories

Maintain a list of every model, its version, its licence and the runtime serving it, whether Ollama or vLLM, with the hardware it runs on. Reviewers ask this because model behavior changes between versions and because vLLM and Ollama both ship frequent releases. Record the date each model was approved and which version of each component was in use during any given period.

Questions teams ask before an air-gapped Glean replacement

What drives the cost of an air-gapped Onyx deployment?

The cost is driven mainly by GPU or accelerator capacity for inference, the infrastructure for the index and database, the connector work for your source systems, and the people who operate the stack. Onyx Community Edition is MIT licensed, so software licensing is not the main line item unless you choose the self-hosted Enterprise Edition, which adds premium features such as RBAC. Inference hardware is usually the largest one-time decision and recurring cost, and its size depends on model choice and concurrent users. Ask any partner to split the quote into hardware, software, connector build, staffing and support so you can see which items recur.

How long does an air-gapped Onyx deployment take?

The timeline depends on scope: how many connectors you need, whether the network needs a hardware refresh, which model servers you run, and how much permission testing your compliance team requires. A lighter chat deployment without a vector database or background workers is simpler to stand up than Onyx Standard, which includes vector and keyword indexing, sync workers, model inference servers, object storage and Redis. Ask for a duration in the written scope, broken into phases such as environment preparation, connector validation, model serving tests and a pilot group. Treat any duration without a stated scope as unverified.

Can our team run an air-gapped Onyx stack without outside help?

Yes, a team with container, database and GPU operations experience can run it, but the work is broader than installing the software. Onyx's documented infrastructure includes Postgres, Vespa or OpenSearch, Redis and model servers, and deployment is possible with Docker, Kubernetes, Helm or Terraform. You also need people to build and maintain connectors, manage permission sync, rotate credentials and handle upgrades. Assess your staff against that list honestly. A fixed-scope engagement from DeployOpen can cover planning, implementation, hardening, backup and documentation, with ongoing support available as a retainer.

Should we choose Ollama or vLLM for our air-gapped hardware?

Choose Ollama when you need a simple local runtime on modest hardware, and choose vLLM when you need high-throughput serving across GPUs or nodes. Ollama runs on macOS, Windows, Linux and Docker, stores models in a directory you can move with OLLAMA_MODELS, and its local API at port 11434 does not require authentication, so it must sit behind your own access controls. vLLM supports NVIDIA CUDA, AMD ROCm and Intel XPU GPUs, x86 and ARM CPUs, and exposes OpenAI-compatible endpoints. It also scales with tensor and pipeline parallelism. Confirm that your accelerators and drivers are supported before you buy, and test your target model on your hardware first.

How are indexing and updates handled without internet access?

Indexing runs inside your network against the connectors you deploy, so the source systems must be reachable from the Onyx services but not from the internet. Onyx's document index, database and processing run in a self-contained set of services, which keeps content within your boundary. Software and model updates have to be staged: you download container images, Helm charts and model weights on a connected machine, verify them, then move them across your approved transfer process. Onyx publishes releases such as v4.9.2 and a Helm chart, and Ollama and vLLM publish frequent releases, so set a documented update cadence and confirm how your transfer process handles them.

What should we ask for in a fixed-scope quote?

Ask for a written scope that lists each connector, each permission sync source, each model and serving engine, the hardware assumptions, and the environments covered. Request separate line items for implementation, connector work, hardening, backup and recovery, documentation, and the pilot group. Ask what is one-time and what recurs, including support, upgrades and any Onyx Enterprise Edition license you choose. Confirm acceptance criteria, such as how search quality will be compared on real queries before cutover. DeployOpen does not publish a price list, so the quote follows from the written scope.

Does an air-gapped deployment help with compliance evidence?

It can keep content, logs and model inference inside your boundary, which simplifies some data residency and access questions. An air-gapped setup does not itself satisfy any framework. DeployOpen prepares systems, controls and evidence for compliance assessments, but independent auditors and certification bodies issue reports and certifications. Ask your assessor which evidence they expect, such as access logs, permission sync records and change records for updates, and plan for those artifacts from the start.

Official sources to check before you decide

Start with the Onyx deployment overview and the Onyx README before you commit to an air-gapped design. The README documents an air-gappable, self-hosted deployment in which the document index, database and processing run inside a self-contained set of services, deployable with Docker, Kubernetes, Helm or Terraform. Read the Onyx deployment overview alongside it and map each listed component (Postgres, Vespa or OpenSearch, Redis, model servers) to a host in your isolated network. Then ask Onyx in writing which connectors, if any, reach out to the internet during indexing, which telemetry or license checks run at startup, and how the Community Edition versus Enterprise Edition split affects features you need offline, such as RBAC and SCIM.

Next, check the model runtime. The Ollama FAQ confirms that a local Ollama instance does not see your prompts, that cloud features can be turned off with OLLAMA_NO_CLOUD=1, and that models are stored in a directory you can move with OLLAMA_MODELS. Confirm that your model files are copied in through an approved transfer process, since the runtime itself does not pull them from a network you control. For vLLM, read the vLLM security docs closely: --api-key does not protect /metrics, /health or several other endpoints, and multi-node communication is insecure by default. Plan a reverse proxy that allowlists only the endpoints your users need, and isolate the node network.

Finally, review the vLLM Kubernetes deployment docs if you run inference on a cluster, and confirm how image and chart updates reach a network with no outbound access. Both vLLM and Ollama ship releases often, so your offline update process needs a named owner, an internal mirror for images and model files, and a test step before each change. Ask each vendor to confirm offline behavior for the exact version you plan to deploy, and record the answers in your architecture decision record. The sources describe design intent. They do not replace a written confirmation from the vendor for your configuration.

How DeployOpen deploys Onyx under an air-gapped constraint

DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure, including isolated networks. For an air-gapped Glean replacement, we start by mapping which sources, models and update paths can exist inside the boundary, then build and document the stack to match that boundary.

How we scope

Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
We inventory the connectors your team needs and confirm which sources can be reached from inside the isolated network.
We confirm model choice between Ollama and vLLM, GPU capacity, and how offline model and container images will be brought in.

How we build

We deploy Onyx with Docker, Kubernetes, Helm or Terraform inside your own accounts, data centers or air-gapped networks.
We configure SSO, SCIM provisioning and permission sync so answers only draw on documents each user can already see.
We place the model endpoints behind a reverse proxy that allowlists only the routes meant for users, and test that the boundary holds.

How we hand over

We document the offline indexing and update procedure, including how connector data, images and model files are transferred and verified.
We deliver runbooks for backups, upgrades, index rebuilds and recovery, written for your own operations staff.
We provide controls and evidence that your team can give to auditors, while independent auditors and certification bodies issue any formal assessment.

How we support

Ongoing support, upgrades and continuous testing are available as retainers after handover.
We can run security testing of the deployed applications, APIs and model endpoints as part of the agreed scope.
Ask any provider to confirm in writing how they will handle offline upgrades, license terms for Onyx Enterprise Edition code, and staff access to logs.