Ollama security hardening

The proxy is where transport encryption, client restrictions and request limits get added to an Ollama server. Ollama's FAQ describes exposing the service through proxies such as Nginx, and the design work is deciding what the proxy accepts before any traffic reaches the runtime. Ollama listens on 127.0.0.1 port 11434 by default and, as the Ollama API authentication documentation notes, the local API at localhost:11434 does not require authentication. That means everything the proxy does for security is the only control between a network client and the model server, so the proxy configuration should be treated as part of the security boundary, not as a convenience layer.

Start by listing the paths and HTTP methods the proxy will forward. The Ollama API exposes generation, chat, model management and administrative endpoints, and these do not carry the same risk. A common design is to forward only the inference paths that the client applications actually call, reject everything else with an explicit error, and deny methods the application never uses. Write that allowlist down as a reviewed artifact, then test it with requests to paths outside the list. Confirm the exact path names and the proxy directives for location and method matching against the Ollama API reference and the Nginx documentation for your proxy version, because path behavior changes across releases.

Certificates are the second design decision. Decide who issues the server certificate (an internal certificate authority, a managed public certificate for a hostname, or a certificate from your platform team), where the private key is stored, and how renewal is triggered. Clients that call the API must trust the issuing authority, so the trust bundle distributed to applications is part of the design. Set a rotation schedule that matches your certificate policy, automate reloads of the proxy after renewal, and test that an expired or revoked certificate causes clients to fail closed. Confirm TLS protocol versions and cipher settings against your organization's baseline and the proxy vendor's current guidance rather than copying old examples.

The proxy must also be the only route to the upstream. If Ollama binds to a reachable interface through OLLAMA_HOST, anyone who can reach that port can bypass the proxy entirely. Keep the runtime bound to the loopback address or to a private interface that only the proxy can reach, and confirm with a port scan from the client network that port 11434 is not reachable directly. Host firewall rules, container network policies or security group rules should enforce this, not the proxy configuration alone. Check the bind address after every restart and upgrade, since a changed environment variable or service file can silently re-expose the API.

Finally, decide what the proxy logs and where those logs go. At minimum, record the timestamp, source address, method, path, response status, and upstream response time for each request, and send them to a store your security team can query and retain. Avoid logging request bodies by default, because prompts may contain personal or confidential data, and Ollama's FAQ says that when it runs locally it does not see your prompts or data. Agree the retention period and access rights with your compliance owner before go-live. Per-user identity in these logs depends on the authentication layer, which is covered in the SSO and identity topic.

The first control for an Ollama server is a rule about where each model comes from, who approves it, and how the approved version is recorded. Ollama pulls models into a local store and loads them on demand, so a model can reach a production host with one command and no review unless your team builds that step in. Decide before go-live which sources are acceptable, which roles can add a model to a production host, and where approvals are kept. Those three decisions shape every later step in this section.

Record enough detail that someone can later answer exactly what ran and where. For each model, capture the model name and tag, the source it was pulled from, the date it was pulled, the host or hosts it was loaded on, and its license. The license matters because model terms differ from the MIT license that covers the Ollama runtime itself, and your legal or compliance team will need to review commercial-use and redistribution terms for each model separately. A simple register in your configuration management system or ticketing tool is enough for most teams, provided every production change points to an entry in it.

Do not assume Ollama gives you a signature or checksum process for every model. Confirm what the registry and each model publisher supply for the specific versions you plan to use, including whether they publish digests you can compare against, and whether those digests are stable across pulls. Document the answer per model family. Where the answer is no, or where your policy requires stronger assurance, the mitigation is to build the control yourself: keep an internal allowlist of approved model names and tags, and keep a copy of each approved model file in storage you control so the exact artifact can be re-verified or restored.

Enforce the allowlist at the host, not only in a document. Limit who can pull models on production machines, and review any change to the model directory, which is set by OLLAMA_MODELS if you have moved it from the default location. Illustrative example: a team pulls a tag on a staging host, compares its recorded details against the register, copies the approved files to internal storage, and only then promotes the same tag to production. Re-pulling a tag later can return a different artifact if the publisher has updated it, so promote the verified copy rather than repeating the pull.

Ask your security reviewer to sign off on the procedure before the first production model is loaded, and retest it whenever you add a new source or change hosts. Plan for periodic review of the allowlist, especially when a publisher releases a new version under an existing tag. The effort for this control is mostly process and recordkeeping, which recurs each time a model is added or updated, so include it in the ongoing support scope when you compare options with a partner.

The hardening work only holds if the people who inherit the server can see what was configured and why. Before go-live, hand over a set of named artifacts, and assign a named owner to each one. A shared team mailbox is not an owner. Ask the implementation partner to confirm in writing who receives each item, who can change it, and where the current copy lives. Without that, the first person who changes OLLAMA_HOST during an incident will not know which other settings depend on it. The configuration record is the most important item. It should list every OLLAMA_ variable set on the host, including OLLAMA_HOST, OLLAMA_MODELS, OLLAMA_NO_CLOUD, OLLAMA_NUM_PARALLEL, OLLAMA_MAX_LOADED_MODELS and OLLAMA_MAX_QUEUE, with its value and the reason for it. Record defaults that were deliberately left unchanged too, because an absent variable is a decision as well. Pair the record with the proxy configuration, the name of the person who owns the TLS certificate, its expiry date, and the renewal process. Store the firewall rules next to it, stating which source addresses may reach port 11434 and which outbound destinations the host is allowed to use. The approved model list needs provenance records for each model: where it came from, the version or digest pulled, who approved it, and the date. The test results should show what was checked before go-live, such as bind address, unauthenticated access from outside the allowed network, and cloud features being off. Finally, name the patching owner, who decides when the Ollama binary and the host operating system are updated and who is accountable if an update is skipped. After go-live, monitor four things. Watch for listener changes, meaning any process bound to a new address or port on the host. Watch for unexpected outbound connections, since a local runtime should not normally reach external hosts unless a cloud feature or a model pull requires it. Track request volume against the queue limit set by OLLAMA_MAX_QUEUE, and against OLLAMA_NUM_PARALLEL, so the owner sees saturation before users do. Review errors logged at the proxy, including authentication failures and upstream timeouts, because they often show misconfiguration or misuse first. Schedule a re-check of the hardening baseline after each new Ollama release, not only after major changes. Releases are frequent, so agree the cadence with the patching owner and confirm how release notes will be reviewed for changes to bind behavior, cloud features or defaults. Version upgrade testing is covered in the upgrades topic, so this section only covers the security re-check. Confirm with the implementation partner what that re-check includes and who signs off on it.