How do you connect Open WebUI to Ollama and OpenAI-compatible APIs without giving users the keys?
You connect Open WebUI to Ollama, OpenAI, Anthropic or any OpenAI-compatible provider by adding each service as a connection in the administrator settings, then deciding which models each user group can reach. Provider credentials stay in the server-side configuration, and users only ever select models that an administrator has exposed to them. The first working chat takes little time. The governance around it, meaning which connections exist, who sees which models and where keys are stored, is where a rollout succeeds or fails.
Three things make this harder than it looks. The first is credential exposure. Connection settings are administrator functions, but the boundary between what the server holds and what a browser or user-level settings page can display needs to be verified in your build, not assumed. Test it with a non-administrator account and confirm that no API key, base URL with embedded credentials, or token appears in responses, page source or exported configuration. The second is model lists. When a connection is added, the model catalog it returns can appear to every user by default. Expect to review that list model by model, hide what is not approved, and then grant access by group so that a new provider does not silently widen what staff can use.
The third is provider behavior. Ollama serves models that run on hardware you operate, while OpenAI-compatible endpoints differ in how they name models, handle streaming, report errors, count tokens and enforce rate limits. Anthropic has its own API shape. A model that works on one connection can fail or return different output on another, and a shared interface hides those differences from users. Ask each provider team which request and response behaviors they depend on, and test each approved model through the interface before it is granted to a group.
Plan for these decisions before go-live. Write down the approved provider list, the owner of each key and its rotation process, and the group-to-model mapping. Confirm with your vendor or your own tests which connection and model-visibility controls exist in the Open WebUI version you deploy, because settings move between releases. Then have a non-administrator confirm what they can and cannot see. Keys that were never shown to users cannot leak from the interface, and approved-only model lists keep the catalog small enough to govern.
Which Open WebUI connections, groups and model settings carry the routing
Routing users to approved models in Open WebUI depends on a small set of connections and access settings working together. Each one decides where a request can go, who can see a model, and which credentials ever leave the server.
Ollama connection endpoints
Each Ollama connection points Open WebUI at a model server, usually on a private address inside your network. Confirm that the endpoint is reachable only from the Open WebUI service and not from user workstations, because a directly reachable Ollama port bypasses every access rule set in the interface. Check which hosts are listed, and remove any test endpoint before go-live.
OpenAI-compatible base URLs and keys
Remote providers are added as OpenAI-compatible connections, each with a base URL and an API key. Record which provider account owns each key, who can rotate it, and where it is stored. Confirm whether the key is visible to administrators only or to any user who opens the connection settings, since that answer determines who can copy a provider credential out of the platform.
Model visibility and group access
Open WebUI lets administrators control which models appear for which groups, so a provider's full catalog does not have to be exposed to everyone. Map each approved model to the groups that need it, then test with an account from each group. Confirm the exact menu and permission names in the features documentation for your release, because the labels have changed between versions.
User-issued API keys
Users can be issued Open WebUI API keys for programmatic access, which is separate from the provider keys held by the server. Decide whether API key creation is allowed for all users or only for specific groups, and whether keys can be restricted to particular models or endpoints. Check the current scope options for your release before writing this into an access policy.
Default model and connection precedence
When several connections serve similar models, the names and default selections determine which backend answers a user's request. Name models so that the provider and version are obvious, and set a default only for models that have been approved for the whole user base. Test a request for each visible model to confirm it reaches the intended connection and not a fallback.
Network paths to providers
Outbound traffic from the Open WebUI service to each provider is the last link in the routing chain. Confirm which egress paths are permitted, whether a proxy is required, and whether the provider endpoint is the one your contract names. Document the allowed destinations so a security reviewer can compare them against the connection list.
Local Ollama, remote OpenAI-compatible or both: which provider mix fits
This table compares three provider setups for Open WebUI: Ollama only, a remote OpenAI-compatible provider only, and a mixed setup with an approved model list per group. Read each row as the operating cost you accept in exchange for the data path or capability you need. The main takeaway is that the mix should follow data classification and model needs, and that every option carries recurring upkeep you should price into the decision.
| Decision factor | Ollama only (local) | Remote OpenAI-compatible only | Mixed with approved model list per group |
|---|---|---|---|
| Best fit | Data that must stay inside your environment, including prompts and retrieved documents | Workloads where model capability or burst capacity matters more than keeping prompts on your hardware | Organizations with several user groups, where sensitive groups stay local and others use remote models |
| GPU capacity | You size and buy or reserve GPUs for peak concurrent use, and capacity is limited to what you run | No GPUs to run, but usage depends on provider rate limits and quotas you must confirm | GPUs sized only for the local group, with remote capacity covering the rest |
| Model updates | Your team pulls, tests and retires model versions and tracks their licenses | The provider changes or deprecates models on its schedule; confirm notice periods in its terms | Two update cycles: local models you manage and remote model aliases you monitor |
| Key management | Few or no provider keys; access control is handled by Open WebUI groups and model permissions | Provider API keys must be stored, rotated and kept out of end-user hands in Open WebUI connection settings | Keys for the remote connection only, scoped to the groups allowed to use them |
| Data egress and terms | Prompts and documents do not leave your network for model inference | Prompts and responses leave your network; review retention, training and region terms with the provider | Egress is limited to the remote group; approved lists must state which data classes may be sent |
| Operational upkeep | Model hosting, GPU monitoring and Ollama version management | Connection monitoring, spend and quota alerts, and key rotation runbooks | Both sets of upkeep, plus a routing policy that must be reviewed when either side changes |
| Main risk | Capacity shortfalls at peak use and slower model turnaround | Unapproved data leaving the environment through a user choosing the wrong model | Misconfigured group mapping that sends restricted users to a remote model |
| Ask before choosing | Which GPUs, how many concurrent users and which models the business needs | Which provider terms cover retention, training and data location for your data classes | Who owns the approved model list and how often it is reviewed |
How to connect Ollama and a remote provider step by step
This sequence connects a self-hosted Open WebUI instance to a local Ollama server and one remote OpenAI-compatible provider, then limits users to an approved model list. It is written for the administrator or platform engineer who builds the first working connections, and each step closes with a check that must pass before the next step begins.
Install Open WebUI with a supported method
Platform engineering installs Open WebUI using Docker, Docker Compose, Helm, or pip install open-webui followed by open-webui serve, the four methods the project documents. Choose the method that matches how your team already runs services, and keep this test instance on a single replica. Confirm the step is complete when the first administrator account can sign in and the web interface loads from the address your network team configured. If the instance will later run several replicas, the scaling guide requires PostgreSQL and Redis, so record that decision now.
Confirm Ollama is reachable from the container
Before adding any connection, check that the Open WebUI container or pod can open a network path to the Ollama endpoint. Inside a container, localhost refers to the container itself, so the address must point to the host or service where Ollama listens. Run a request from inside the same container or pod to the Ollama API and confirm it returns the models installed on that server. The step is done when the model list from inside the runtime matches what the Ollama host reports and firewall or network policy allows that path.
Add the Ollama connection
In the administrator settings, add an Ollama connection and enter the endpoint you confirmed in the previous step. Keep the connection visible only to administrators during testing so end users cannot change the address. Save the connection and refresh the model list. The step is complete when the models shown in the model picker match the approved local set, and any model outside that set is left for the next step to hide.
Add the OpenAI-compatible connection securely
Add a second connection of the OpenAI-compatible type. Enter the base URL from the provider's documentation and paste the API key into the server-side connection field, where administrators can manage it and end users cannot read it. Ask the provider which base URL and key scope apply to your account, since compatible services can differ in their endpoints. The step is complete when the save succeeds, the provider's model list loads, and the key does not appear in any page that a standard user can open.
Restrict models to the approved set
Limit which models end users can see. For each connection, hide every model that is not on the approved list, and grant model access through groups so future changes follow your role structure. Use the role-based access controls the platform documents, and confirm the permissions in the admin settings. The step is complete when a test user in a standard group sees only the approved models from both connections, and an administrator can still see the full set for review.
Test a chat on each connection
Run a chat on an approved Ollama model and on an approved remote model. Then send a request through the OpenAI-compatible connection with a deliberately wrong key, which is an illustrative check and should be done on a test account. A correct setup returns a readable answer on both working connections and a clear authentication error on the bad key. Confirm that the key value does not appear in the error shown to users or in the logs you review, then restore the valid key and record all three results for the handover.
How model access by group keeps provider keys out of user hands
Users should be able to select only the models your organization has approved, and provider keys should never leave the server. Both goals depend on the same design: group membership decides which models a user can see, and provider connections are configured by administrators on the server side. Start by listing the groups your users actually belong to, such as general staff, a research team or a support desk, and the approved models each group needs. Then set per-resource permissions on each model so that a group sees only its own models in the selector. Test this design before rollout, because a model that is hidden in the menu can still be reachable through other routes if permissions are set only in the interface.
Keys belong in the server-side connection configuration that Open WebUI uses to reach Ollama, OpenAI, Anthropic or any OpenAI-compatible provider. A connection entry holds the base URL and the credential, and only administrators should be able to create, edit or view it. The risk is usually not the connection itself but the places where a key can be copied into a less protected setting, such as a user-owned connection, a shared prompt, a tool or a function. Decide whether users may add personal connections at all. If they may, each personal connection carries its own key outside your control. Confirm which connection and model settings are visible to non-admin roles in your release, and make sure those pages display the provider endpoint without revealing the secret.
Where the provider allows it, issue the OpenAI-compatible key from a dedicated project or account used only by this deployment. A dedicated key limits the damage if a credential is exposed, because you can revoke or rotate it without breaking other applications, and it lets you attribute usage and spend to this platform. Ask your provider which scoping options exist, such as project-level keys, model restrictions, spend limits and rotation tools, and record the answers in your design. Ollama does not issue keys in the same way, so for local models the controls are network access to the Ollama service and which models are pulled onto it. Confirm both sides before you finalize the group-to-model mapping.
Test the design from the user's side before go-live. Create a test account in each group, sign in as each one, and check that the model selector shows only the approved models. Open every settings page a non-admin role can reach and look for the provider key in the page content, in form fields and in the page source. Then use the browser developer tools to inspect the responses the interface receives, and check that no response contains the key. Repeat the check with a user API key, requesting the endpoints that list models and connections, and confirm the provider credential is not returned. Document the results with the release version under test, and repeat the tests after every upgrade, because settings visibility can change between releases.
What drives effort for multi-provider model routing
This table lists the factors that change how much work a multi-provider Open WebUI setup takes and how to keep each one contained. Read each row as a question to settle before scoping, since the controls named in the right column are the levers that reduce effort. The main takeaway is that the number of provider connections and the number of user groups matter more than the number of models alone.
| Driver | Why it changes effort | How to keep it down |
|---|---|---|
| Number of providers and connections | Each connection needs its own base URL, credential, model discovery check and failure handling, so effort grows with every added provider. | Limit connections to providers the organization has approved, and add a new one only when a group has a documented need that existing connections cannot meet. |
| Number of models exposed | Every model a user can select needs an owner, a documented purpose and a test pass, and the list grows with each provider added. | Publish one shared approved model list and hide unlisted models from user groups, so review happens once rather than per team. |
| Groups needing different model sets | Per-group model access multiplies the permission rules to set, verify and document, and each change touches more users. | Start with a small set of groups mapped to a few model tiers, then add groups only when access rules cannot be met by an existing one. |
| Ollama on dedicated GPU hosts | Local inference adds host sizing, driver and GPU maintenance, model pull management and capacity planning that remote APIs do not require. | Confirm the GPU hosts and model storage needs with your infrastructure team before scoping, and keep the local model set small enough for the hardware. |
| Remote keys issued per team | Separate keys per team mean more secrets to create, rotate, revoke and audit, and more places where a key can leak. | Issue keys per provider rather than per user, store them in your secrets manager and keep them out of end-user settings where your configuration allows. |
| Network paths and egress rules | Each provider endpoint needs an approved outbound route, firewall or proxy rule and DNS entry, and each path needs a change request when it moves. | Route all provider traffic through one egress point or proxy, and list every required destination in a single rule set that your network team approves once. |
| Testing each provider's behavior | Providers differ in model names, streaming, error codes, token limits and tool-call support, so each one needs its own checks before users rely on it. | Use one standard test script that checks model listing, a short completion, a streamed response and a deliberate error case, and rerun it after every provider or model change. |
Keeping provider keys, API keys and model access under control
An Open WebUI deployment that routes users to Ollama and remote OpenAI-compatible providers concentrates provider credentials and model spend in one place. The cards below cover where those keys live, who can change connections, how sign-up and user API keys are scoped, which network paths are open, and what the license allows if the rollout is branded. Identity and general hardening are covered in the SSO and security hardening topics.
Store provider keys in deployment secrets
Provider keys for OpenAI-compatible endpoints should be injected from your secrets manager or the orchestrator's secret store, not typed into the admin connection settings where they can be read back by anyone with admin access. Confirm with your Open WebUI version how connection credentials are persisted in the database and whether they are masked in the admin interface. Plan a rotation procedure that updates the secret and restarts or reloads the affected replicas.
Limit who holds the admin role
Connection settings, model visibility and user management all sit behind administrative roles, so every additional admin can view or replace provider credentials. Keep the admin group small, name a named owner for each provider connection, and use group-based permissions for everyone else. Review admin membership on a fixed schedule and after staff changes, and confirm which settings each role can modify in your release.
Disable open sign-up before go-live
If sign-up is left open, anyone who can reach the login page may create an account and consume paid model capacity through the connected providers. Turn off self-registration, decide whether new accounts are provisioned through SSO or SCIM, and test that an unauthenticated request cannot create a user. Confirm the current setting name and default in the admin documentation for your target version.
Scope user API keys
Open WebUI can issue API keys to users, and each key carries the access of the account that created it. Decide whether end users may create keys at all, which models those keys can reach, and how a departing user's keys are revoked. Ask your team to test that a key issued to a standard user cannot call admin endpoints or read provider settings.
Control outbound access to provider endpoints
The Open WebUI server, not the browser, calls Ollama and remote APIs, so egress rules must allow only the endpoints you have approved. Restrict outbound traffic from the application network to the named provider hosts and to your internal Ollama service, and log denied connections. Confirm with each provider which hostnames and regions it uses, since these can change.
Review the license before branding
Open WebUI code from v0.6.6 onward is distributed under the Open WebUI License, which restricts altering or removing branding in deployments above 50 users in any rolling 30-day period unless an enterprise license or written contributor permission applies. Review the license text with counsel if you plan to rebrand, and request enterprise terms in writing, including what white-labeling covers and the support scope. Pricing is quoted by Open WebUI's sales team against an expected seat count.
How to test that Ollama routing and key isolation hold
The routing and key isolation design is only as good as the tests that show it works. Before go-live, and again after every Open WebUI upgrade, run a fixed set of test cases against the live configuration, with a standard user account rather than an administrator. Administrators see more models and settings than end users, so tests run as an admin will miss the failures that matter most. Prepare at least one standard user in the group that the approved models are assigned to, one standard user outside that group, and an administrator who can confirm the configuration independently.
The first set of cases checks routing. Each approved model should answer from the provider it is meant to use: a model registered under the Ollama connection should be served by your Ollama host, and a model registered under an OpenAI-compatible connection should be served by that remote endpoint. Confirm this in the administrator's connection and model settings, then send a short prompt from the standard user and check the response metadata or the provider logs on each side. Unapproved models must not appear in the model selector for the standard user, and a direct request for an unapproved model identifier should be refused. Record the model name, the provider log entry and the timestamp for each approved model.
The second set checks key isolation. A standard user must not be able to view a provider key in the interface, in a settings export, in a browser network response or through the API. Export the configuration as an administrator, then search the output for the key prefix, and repeat the check from the standard user's session and browser developer tools. Next, enter a deliberately invalid key for a test connection and confirm that the error shown to the user is a controlled message without the key, the full upstream URL or stack traces. Then point the Ollama connection at an unreachable address and confirm the user sees a generic failure, with no internal hostnames or IP addresses in the message. Confirm with your vendor and your own logging policy what the server writes to its logs in each case.
The third set checks access removal. Remove the test standard user from the approved group, then confirm that the approved models disappear from their selector and that a direct request fails. Confirm that existing sessions are also cut off, which depends on how token revocation is configured in your deployment, so verify it rather than assume it. Re-add the user and confirm access returns only through the group. Store every result in a test record with the test ID, the Open WebUI version and configuration hash, the tester, the expected result, the observed result, the evidence reference and the date. A security reviewer should be able to trace each control to a result without asking follow-up questions.
Repeat the full set after each Open WebUI upgrade and after any change to connections, groups or model visibility, because upgrades can change defaults and interface behavior. Keep the previous evidence so the reviewer can compare results between versions. Plan the retest window before the upgrade starts, assign a named owner for each test, and treat any regression in key isolation as a release blocker. Ask your implementation partner to include this test pack in the handover, and confirm whether ongoing retesting sits in the support scope or is a separate retainer.
What to monitor, back up, rotate and document after go-live
Once users are on Open WebUI, the connection layer needs routine attention: provider errors, Ollama host health, stored credentials and the approved model list. These items are linked because a change in any one of them, from a failed host to a retired model, shows up to users as a broken or unexpected chat. Assign an owner to each before go-live so none of them depends on memory.
Watch provider errors and Ollama health
Track error rates and failed requests for each connected provider, including remote OpenAI-compatible endpoints, and track availability of each Ollama host. A single model failing often looks like a general outage to users, so alerts should name the connection and model involved. Check that the Open WebUI logs and your monitoring agree on what counts as a failed request, and confirm the alert routes to the team that can rotate a key or restart a host.
Back up the connection configuration database
Provider endpoints, model settings and user and group records live in the Open WebUI database, so a backup of that database is the main recovery path for connection setup. If the deployment uses PostgreSQL, schedule backups with tested restores into a separate environment. Confirm whether provider credentials are stored encrypted in your version and where the encryption key is kept, because a backup without its key may not restore the connections.
Rotate provider keys on a schedule
Provider API keys should be rotated on a fixed schedule and immediately after any staff change involving people who held or could read them. Keep keys in the Open WebUI admin connection settings rather than in shared notes, and confirm that end users never see the key value through the interface or API responses. After each rotation, run a test request against every connected provider to confirm the new key works before revoking the old one.
Re-test model lists after each upgrade
After every Open WebUI upgrade, compare the model list each connection returns with the approved catalog. New versions can change how models are discovered, labeled or filtered, and a model that was hidden may reappear or an approved model may disappear. Run a short set of prompts against each approved model in a staging copy before the upgrade reaches users, and keep the results with the change record.
Review the model catalog on its own cadence
Provider-side changes can alter model behavior without any Open WebUI release, for example when a provider adds a model, changes a version alias or retires an older one. Set a catalog review interval separate from the software upgrade cycle, and check each provider's release notes during that review. The catalog document should list each approved model, its provider or Ollama host, the owner who approved it and the steps required to add, change or remove it.
Document owners and change steps
Record who owns each provider connection, each Ollama host, the key rotation process and the approval for catalog changes. Write the change steps as a short runbook covering adding a model, rotating a key, moving a host and rolling back a failed change. Review these owners whenever staff change roles, because stale ownership is the most common reason a rotation or alert is missed after go-live.
What do teams ask before hiring help for Ollama and OpenAI-compatible setup?
How much effort does an Ollama and OpenAI-compatible setup take?
Effort depends mostly on how many provider connections, models and user groups the setup must cover. A single local Ollama host with one approved model is a small job. Adding a remote OpenAI-compatible API, a second provider, per-group model access and API key handling adds design and testing work for each connection. Other drivers include where Ollama runs, whether GPU hosts are shared, the network paths between Open WebUI and each endpoint, and whether sign-in uses SSO or local accounts. Ask any partner to break the effort into provider connections, model approvals, access rules and verification steps so you can compare the scope line by line.
Can our team set up Open WebUI with Ollama without a partner?
Yes, many engineering teams can complete a single-host setup in-house if they can run Docker or Python services, manage GPU drivers and read the provider documentation. Open WebUI's documented install methods include Docker and Docker Compose, Kubernetes with Helm, and pip install open-webui followed by open-webui serve. The harder parts are controlling which users reach which models, keeping provider keys server-side, testing behavior when a provider fails and documenting the result for audit. Consider outside help when those controls must be designed, reviewed and evidenced before go-live, or when your team lacks capacity to run the setup after handover.
What does the work need from our side before it starts?
Most of the delay in these projects comes from inputs that sit outside the configuration itself. Plan to provide:
- Provider accounts and keys for each remote provider, issued under your procurement and key management process
- Host access to the GPU machines or Ollama servers, including network paths and a change window
- Identity decisions: whether users sign in with local accounts, OIDC or LDAP, and who owns group mapping
- An approved model list, with an owner for each model and the groups allowed to use it
- A named decision-maker who approves scope changes and signs off test results
How long does an Open WebUI and Ollama setup take?
Duration follows scope and environment, so no single timeline applies. A single-host setup with one local model and local accounts moves faster than a build with several providers, SSO, per-group model access, a shared database and a test cycle with business users. Your approvals, network changes and GPU capacity also set the pace. Ask for a timeline with milestones for provider connections, model approval, identity integration, testing and go-live, and check which milestones depend on your teams or vendors. Waiting on provider accounts or host access often sets the critical path more than configuration work does.
Does a fixed-scope agreement suit an Ollama and OpenAI-compatible rollout?
Often yes, when the scope is settled before work starts. Most DeployOpen engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work begins. This setup suits that model when the provider list, model list, identity method and hosting target are known. It suits that model less well when the team has not yet decided which models to approve or which providers to use, because those decisions change the work. If you expect the setup to evolve after go-live, ask about a retainer, which DeployOpen also offers, for ongoing support, upgrades and continuous testing.
What drives the cost of this work, and does DeployOpen publish prices?
DeployOpen does not publish a price list. A quote follows from a written scope. The main cost components are planning, provider and identity configuration, access control design, testing and documentation. Costs rise with the number of provider connections, models under approval, user groups, identity integrations and environments such as test and production. Some costs are one-time, such as design and go-live. Others recur, such as model and API usage, GPU capacity and any support or upgrade retainer. Ask each bidder to separate one-time from recurring items and to state which provider and hosting costs you pay directly.
Does the Open WebUI license change how we set up Ollama and remote providers?
It can, mainly through branding and user count. Since v0.6.6, Open WebUI code is distributed under the Open WebUI License, which includes a branding protection clause. Code through v0.6.5 remains BSD-3-Clause. The license forbids altering or removing branding in deployments that exceed 50 users in any rolling 30-day period, unless the operator holds an enterprise license or written contributor permission. Enterprise pricing is not published and is quoted after contacting enterprise sales with an expected seat count. Read the Open WebUI license documentation and the enterprise documentation, then confirm your user count and branding plans with the vendor before scoping.
How should provider keys stay out of end users' hands in Open WebUI?
Store provider credentials in server-side connection settings that only administrators can manage, and limit who holds admin roles. Confirm in your target version how connection credentials are stored, displayed and exposed to users, because this varies by release and configuration. Pair that with per-group model access so users see only approved models, and issue API keys to people or services under your own policy. Test by signing in as a standard user and checking the interface, network responses and logs for secrets. Document a rotation process for each provider key before go-live.
How DeployOpen scopes, builds, hands over and supports this work
DeployOpen plans, implements, documents and supports self-hosted Open WebUI inside your own infrastructure, connecting it to local models through Ollama and to remote OpenAI-compatible APIs. Engagements are fixed-scope, with the provider connections, routing rules and key handling agreed in writing before work starts.
How we scope
- Inventory which Ollama hosts and remote OpenAI-compatible providers the rollout must reach, and who approves each model.
- Define which users reach which models, and where provider API keys are stored so end users never see them.
- Record the written scope, deliverables and timeline, and confirm whether ongoing support is needed as a retainer.
How we build
- Deploy Open WebUI with Docker, Docker Compose, Kubernetes with Helm or pip, matched to your hosting standard.
- Configure Ollama and OpenAI-compatible connections, model visibility and group-based access to approved models.
- Test model routing, key isolation and failure behavior when a provider is unavailable before users are onboarded.
How we hand over
- Document the connection settings, model catalog, access groups and where each provider credential is held.
- Walk your operators through adding a model, rotating a provider key and checking connection health.
- Deliver the runbook and configuration records so your team can operate the deployment independently.
How we support
- Offer ongoing support, upgrades and testing as a retainer, scoped in writing alongside the original engagement.
- Review each new Open WebUI release against your provider connections before upgrading production.
- Confirm with your team which support hours, response paths and change-control steps apply after go-live.
