Migrate from Glean to Onyx

Glean Hosted runs on Glean's Google Cloud environment, and Glean remains responsible for infrastructure and application management, including upgrades, patching and monitoring. Moving to Onyx means your team takes over that operating role, so the migration plan must start with a clear picture of which Glean-managed duties will become yours.

• Inventory connectors first: List every source Glean indexes today and confirm each one has an Onyx connector in the connectors directory, since a missing connector is a gap you must fill or accept.
• Re-create permission sync per source: Test that Onyx honors each source's document permissions before any user sees results, because search quality means little if access boundaries differ.
• Plan the hosting account: Decide whether Onyx runs in your own cloud account or on-premises, since Glean Hosted does not give you that choice.
• Assign operating ownership: Name who handles upgrades, patching and monitoring, and document the Glean behavior you are replacing.
• Export usage history: Confirm what query history and analytics Glean exposes so you can keep a comparable baseline during the pilot.

Before renewal, confirm with Glean and your own environment exactly which data, logs and admin exports you can retrieve, and run the pilot group on the same queries you use today.

Glean's Customer Hosted model, formerly called Cloud-Prem, runs in your own GCP or AWS account, but Glean still manages infrastructure and application upgrades and patching. Migrating to Onyx moves that operating responsibility to your team or a partner, so the change in day-to-day work is larger than the change in hosting location.

• Reuse the cloud account: Onyx can be deployed with Docker, Kubernetes, Helm or Terraform, so existing Terraform and Kubernetes practices in your account carry over.
• Replace managed services deliberately: Onyx Standard uses Postgres, Vespa or OpenSearch, Redis, model servers and object storage, so decide which of these you run yourself and which you already operate.
• Choose Onyx Lite for narrower scope: If you do not need the vector index and background sync workers, Onyx Lite covers chat and agents without them, which lowers what you must operate.
• Verify SSO and provisioning: Confirm your identity provider works with Onyx through OIDC, SAML or Google OAuth and that SCIM provisioning matches your joiner and leaver process.
• Review enterprise features: Role-based access control and query history auditing sit in the Enterprise Edition, so confirm they meet your requirements before choosing a license path.

Ask your team to map each Glean-managed function to a named owner, then confirm the Onyx edition and feature set in writing.

An air-gapped requirement rules out any model path that sends prompts or documents to an external API, and Glean's Customer Hosted model still depends on Glean-managed services in GCP or AWS. Onyx documents an air-gappable deployment in which the index, database and processing run in a self-contained set of services, so the model path becomes the central decision.

• Pair Onyx with local models: Ollama or vLLM provide model serving inside the isolated network, so no prompt leaves the enclosure.
• Plan updates offline: Container images, Helm charts and model weights must be transferred through your approved process, so define that workflow before the pilot.
• Isolate vLLM nodes: The vLLM security docs state that inter-node communication in multi-node runs is insecure by default and must sit on an isolated network.
• Restrict endpoints: The --api-key option does not protect /metrics, /health or /invocations, so place a reverse proxy in front that allowlists only user-facing endpoints.
• Confirm connector access: Connectors that need internet reachability to SaaS sources may not work in an enclosed network, so confirm the source type for each connector.

Confirm with the Onyx and model vendors which components can operate with no outbound connections in your environment before committing to a timeline.

Using hosted proprietary models through Onyx keeps the retrieval, permissions and chat layer under your control while sending prompts to a model provider such as OpenAI, Anthropic or Gemini. This path avoids running GPUs but adds a provider contract, data handling terms and a per-token cost that scales with usage.

• Check provider data terms: Review each provider's retention and training terms, since Onyx does not change what the provider does with prompts.
• Set spend controls: If you route through a gateway, LiteLLM virtual keys support per-key, per-user and per-team budgets and model restrictions, which need a PostgreSQL database.
• Plan provider failover: LiteLLM's router supports retries, cooldowns and priority ordering, which you configure rather than receive by default.
• Confirm residency: Verify the region where each provider processes inputs, because residency options differ by vendor.
• Budget the usage: Ask providers for their current pricing in writing and model expected query volume from the pilot, not from published estimates.

Confirm the provider list, data terms and budget model with your procurement and privacy teams before connecting any production source.

Self-hosted models keep prompts and retrieved content inside your environment, but they make you responsible for serving, scaling and securing the inference layer. Onyx connects to Ollama, LiteLLM or vLLM, and each has configuration that must be set deliberately rather than left at defaults.

• Serve with OpenAI-compatible endpoints: vLLM started with vllm serve exposes /v1/chat/completions and /v1/embeddings, which Onyx can call once the endpoint is reachable.
• Size accelerators and parallelism: vLLM uses tensor parallelism within a node and pipeline parallelism across nodes, so GPU count and topology drive your hardware plan.
• Gateway with LiteLLM: LiteLLM needs PostgreSQL as its primary database and Redis 7.0 or newer once more than one proxy instance runs, and its master key, which starts with sk-, belongs in a secret manager.
• Monitor serving: vLLM exposes Prometheus metrics at /metrics, which should be scraped on an internal network only.
• Enable SSO carefully: LiteLLM SSO is free for up to 5 users, and Enterprise licensing is required beyond that, so confirm the user count before you choose the gateway configuration.

Test throughput and latency with your own query set on the planned hardware before committing to a model size, and confirm each license and support obligation in writing.