Self-hosted document editing with an on-premises AI assistant

Teams that want writing help inside their browser editor without sending documents to a public AI service need to decide where the model runs, who can use it and what gets logged. This page covers those decisions for Collabora Online.

On this page

Self-hosted document editing with an on-premises AI assistant: why the design starts with data flow

If your team wants writing help inside the editor but cannot send document content to a public AI service, the answer starts with Collabora Online 26.04, which adds an optional AI assistant across Writer, Calc and Impress. The assistant is off by default, and administrators choose the AI provider and whether that provider is hosted externally or on their own infrastructure. The AI decision is therefore a data-flow and access decision, not a feature toggle: before anyone enables anything, you need to know which component sends text to which endpoint, over which network path, under whose credentials, and what gets written to logs. A practical short answer for the platform layer has three parts. Collabora Online remains the browser editor that handles document sessions through WOPI. The model runs on a self-hosted endpoint, such as *Ollama*, a local runtime that serves its REST API on port 11434 and offers OpenAI compatibility, or another OpenAI-compatible service you operate. *LiteLLM* is an optional gateway in front of that endpoint, adding virtual keys, budgets, model restrictions and audit logs where your size and governance needs justify another component. Each of these pieces needs a documented owner, a network rule and a logging decision. The default settings of Ollama matter here. It binds to 127.0.0.1 on port 11434 unless OLLAMA_HOST changes that, and its local API does not require authentication. Once you expose it to other hosts through a proxy, that proxy becomes the access control point and should allow only the editor's service account. Confirm with your Collabora Online version and your chosen model runtime that the assistant calls only the endpoint you configured, and test that outbound traffic from the editor and model hosts is blocked to any external AI service.

Which requirements the AI assistant has to meet

This table maps four requirement areas to what each means for the build of an on-premises assistant in the editor. Read each row as a question to answer before enabling the feature, and treat the build notes as the minimum for a regulated deployment. The main takeaway is that prompts and responses can become records, data and audit subjects, so logging and routing decisions must be made up front.

RequirementWhat it means for the buildWhat to confirm
SEC Rule 17a-4(b)(4) retention for broker-dealersIf AI-generated text is used in business communications, keep the originals and copies your retention policy treats as records for at least three years. Decide whether prompts and generated drafts are records, then store them in a WORM or equivalent retention store you control.Whether the assistant keeps prompt and response history at all, and how a compliance team can export it
HIPAA 45 CFR 164.312(e)(1) transmission securityPrompts can carry ePHI from the document, so the path from editor to model endpoint must be encrypted and limited to hosts you approve. Allow only internal model endpoints and block outbound calls to external AI providers at the network layer.Whether the editor-to-AI connection uses TLS end to end and which outbound hosts it calls
HIPAA 45 CFR 164.312(b) audit controlsRecord who invoked the assistant, which document and user group were involved, the model used and the time. Keep those audit events in your central log platform with access restricted to security staff.Whether the assistant logs prompt text, response text or only metadata, and whether those logs can be disabled or redacted
Default-off configuration and documented provider choiceCollabora Online 26.04 ships the assistant off by default, and administrators choose the AI provider and whether it is hosted externally or on their own infrastructure. Keep the setting off until the approved provider and hosting location are written into your change record, and restrict who may change it.Which settings control enablement, provider and hosting location, and whether changes are themselves logged
Model traffic kept on infrastructure you controlHost the model endpoint in your own environment, such as a runtime like Ollama bound to an internal address, or a gateway like LiteLLM with keys and budgets you manage. Document every host that can receive prompts and check it during each review.Whether the chosen provider sends any data, telemetry or update requests externally, and how to disable them
Access limits on who may use the assistantLimit the feature to groups with a documented need, such as editors of approved document classes, and keep read-only users out of AI actions. Review the group list when roles change.Whether assistant access can be scoped per group or per document type in your deployment

Choosing where the model runs

Ollama is an open source runtime for running large language models locally, licensed under MIT and available for Linux, macOS, Windows and Docker. Placed on a dedicated GPU or CPU host inside your network, the model and the prompts it receives stay on that host. Ollama's documentation states that when it runs locally it does not see your prompts or data. The default configuration is the simplest to reason about, but it is also the least protected, so the network design needs deliberate work.

  • Default bind address: Ollama listens on 127.0.0.1 port 11434, so only processes on the same host can reach it by default.
  • No authentication on localhost: The local API does not require a key, so any process on that host can send requests. Restrict local accounts and the editor's service account accordingly.
  • Changing the bind address: The OLLAMA_HOST environment variable changes where the server listens. Exposing it to other hosts should go through a reverse proxy such as Nginx, which can add TLS and authentication.
  • Disabling cloud features: Set OLLAMA_NO_CLOUD=1 or use the server configuration file to turn off cloud features on the host.
  • Model storage: Models are stored in a per-platform directory, which OLLAMA_MODELS can relocate to an encrypted volume.

Confirm with your platform team which host runs the model, which network segments can reach port 11434, and whether the proxy layer enforces authentication before the endpoint is shared.

Which platforms fit an on-premises AI assistant and why

An on-premises writing assistant in a document editor is a chain of three parts: the editor, a model runtime and a gateway that controls which models are reachable and who can use them. Each part has its own license, its own support model and its own logging surface, so each needs an explicit decision. The cards below describe the components we would evaluate first and what to confirm before choosing them.

Collabora Online as the editor

Collabora Online is the browser editor that hosts the optional assistant, and its server code is MPL-2.0. Collabora 26.04 adds the assistant across Writer, Calc and Impress, and it stays off until an administrator enables it. Administrators choose the provider and whether it runs externally or on their own infrastructure, so confirm that the self-hosted option is the one you configure. The free CODE edition is suited to testing, not production. For production, the Business plan is listed at 3 euros per user per month for up to 99 users, with signed security updates and long term support.

Ollama as the local model runtime

Ollama runs open models on your own hardware under MIT and exposes a REST API on port 11434 with OpenAI compatibility. By default it binds to 127.0.0.1 and its local API requires no authentication, so the editor should reach it only through a controlled network path and a proxy you manage. Concurrency is a real constraint: OLLAMA_NUM_PARALLEL defaults to 1 request per model and OLLAMA_MAX_QUEUE defaults to 512 queued requests. Its Enterprise plan adds model access controls and cost budgets per user or API key, which you should compare with your gateway controls before buying.

LiteLLM as the model gateway

LiteLLM is a Python SDK and proxy that presents more than 100 providers through one OpenAI-style request format, so the editor integration stays the same when the model changes. Its core is MIT, but content under the enterprise/ directory uses a separate license, so confirm which files you deploy. Virtual keys require PostgreSQL and support per-key, per-user and per-team budgets and model restrictions. Running more than one proxy instance requires Redis 7.0 or newer for shared rate limits and router state. Single sign-on is free for up to 5 users from v1.76.0; audit logs, organization and team RBAC are Enterprise features.

Model selection and output quality

Choosing the model is outside the scope of this page and should not be decided from benchmarks alone. Writing quality, language coverage, hallucination rate and the handling of confidential terms all depend on your own documents and prompts. Build a test set from real internal drafts in a closed environment, score the outputs with the reviewers who will use them, and record the model version and runtime version for each result. Confirm the license terms of each model you intend to run, since the runtime license does not govern model weights.

Operational ownership and cost split

Costs fall into three buckets: editor subscription or support, the model hardware or cloud spend and the staff time to run the gateway, database and logging. The editor and gateway licenses are usually one-time setup plus recurring support, while hardware and storage recur with usage. Ask any implementation partner to quote each component separately, state who operates the gateway and where its database and Redis run, and agree in writing how security updates reach each part. Scope the work in a fixed written statement of work before any build begins.

What to log and restrict before anyone clicks the assistant

Decide who may use the assistant, which models each caller may reach, and what gets recorded before the feature is switched on for anyone. Collabora Online 26.04 ships the assistant off by default, so the first control is administrative: keep it disabled in the deployed Collabora Online configuration until the access model below is written down and tested. Then restrict the group allowed to use it, separate from the group allowed to edit, since many staff who can open a document should not be able to send its contents to a model. Confirm in the deployed version how Collabora scopes the assistant to users and documents, because the product's admin settings, not the editor, decide that boundary.

On the model side, put a gateway between the editor and the model endpoint. LiteLLM virtual keys can be limited to specific models and carry per-key, per-user and per-team budgets and TPM and RPM limits, which keeps one team from consuming a shared model and gives finance a spend ceiling. Those keys need a PostgreSQL database. Audit logs, organization and team RBAC, and SSO beyond five users are Enterprise features according to LiteLLM's documentation, so confirm the license tier you need before designing around them. Ollama binds to 127.0.0.1 on port 11434 without authentication by default, so keep it unexposed and reachable only from the gateway.

Logging should answer four questions for any request: who asked, which document was open, which model answered, and what volume was used. Whether prompt and response text is recorded is a separate decision, and it is where HIPAA and SEC expectations apply. The HIPAA Security Rule requires audit controls that record and examine activity in systems containing ePHI under 164.312(b), so a clinical or patient-facing document can make the assistant's request log part of that evidence. SEC Rule 17a-4(b)(4) requires broker-dealers to preserve copies of business communications for at least three years, which may reach assistant exchanges if they are treated as business records. Ask counsel and your records team to decide retention for prompts and responses, and confirm what Collabora and Ollama log in the exact versions you deploy.

Rolling out the assistant in phases

This sequence moves the assistant from a documented decision to a production feature that can be switched off in minutes, with each phase closing on a written output. It keeps the assistant off until the data rules, the model endpoint and the logs have been checked, so the team can show what was enabled, for whom and on which infrastructure.

  1. Classify data and approve use cases

    The document owner, security lead and privacy or compliance lead agree which document types may be sent to the assistant and which may not, such as personal data, regulated records, legal privilege or unreleased financial results. The output is a written list of allowed document types, the groups allowed to use the assistant and the business reason for each. Confirm the list with your own legal and records teams, because the answer depends on your regulations and contracts, not on the software.

  2. Run a pilot on a test server

    The platform team deploys Collabora Online on a test document server connected to a non-production file platform, with the assistant disabled. Testers open sample documents from the approved list and confirm that editing, saving and sharing behave normally. The assistant is then enabled for a named test group only, and the team records which features were used and any unexpected network calls. The output is a signed pilot record that states the configuration tested and the findings.

  3. Harden the self-hosted model endpoint

    The platform team places the model runtime on infrastructure you control and confirms it listens only where intended. For Ollama, the default bind address is 127.0.0.1 on port 11434, so any wider exposure should go through a reverse proxy with TLS and access controls. Cloud features are turned off with the documented setting, outbound egress from the model host is restricted to approved destinations, and any API keys are held in a secret manager rather than in configuration files. The output is a hardening checklist with evidence from each check, such as a port scan result and a blocked outbound connection test.

  4. Set logging and access review

    The security team defines which events are recorded for the assistant, such as which user invoked it, which document and time, which model and endpoint answered, and whether a request was rejected. Logs are sent to your central log platform with retention that matches your records policy, and access to the logs is limited to named reviewers. Where a gateway such as LiteLLM sits between the editor and the model, its per-key and per-user usage records can supply part of this trail. The output is a logging specification and a first access review showing who can read, change and disable the assistant. Confirm with your auditors which log fields they expect.

  5. Enable production with a tested rollback

    The change owner enables the assistant in production for the approved groups only, using the same configuration that passed the pilot. Before go-live, the team rehearses the rollback: turning the assistant off in the editor configuration, blocking the model endpoint at the proxy, and confirming that documents still open and save. The rollback steps are written into the runbook with the named person who can execute them. The output is a change record with the enablement scope, the rollback test result and the date of the first production review.

  6. Review and extend the scope

    After production use, the owners compare the logs and access review with the approved use-case list and record any exceptions. Requests to add document types or groups go back through the classification step, and changes to the model, provider or endpoint go through the hardening checklist again. Updates to Collabora Online and the model runtime are tested on the pilot server before production. The output is a recurring review record that confirms the assistant still matches the approved scope, or a documented decision to narrow it.

Evidence auditors ask for and which component produces it

What proof shows the AI assistant is off by default?

The strongest proof is the saved configuration of the editor server, exported and kept under version control, with the setting that disables the assistant and a change history showing who turned it on and when. Collabora Online 26.04 ships the assistant off by default, so the record should show that the default was never overridden or that any change went through an approved change ticket. Collect the configuration file from the document server, the container image tag or package version in use, and the access log of administrative actions from your deployment tooling. Confirm the exact configuration key and change-tracking method with Collabora documentation and your own environment before you build the evidence pack. DeployOpen prepares this evidence, but auditors issue any assessment or attestation.

How do I document which AI provider is used and where it runs?

Keep a provider and hosting record that names the model, the runtime, the endpoint address, and whether inference happens on your infrastructure or with an external service. Collabora lets administrators pick the AI provider and choose between external hosting and their own infrastructure, so the record should capture that choice explicitly. For a self-hosted model, the runtime record usually comes from Ollama, which runs locally and states that it does not see prompts or data when run this way, and the configuration showing that cloud features are disabled through OLLAMA_NO_CLOUD=1 or the server configuration file. Include the model version and data location for any hosted option, and ask the provider for its data processing terms. Auditors will test whether the record matches what the network actually does.

What does the gateway access list and key inventory show?

It shows every person, team, service account and API key allowed to reach the model, along with the models each key may call and the budgets or rate limits attached to it. When the gateway is LiteLLM, virtual keys stored in PostgreSQL carry per-key, per-user and per-team budgets, TPM and RPM limits, and model restrictions, so the export of that data is the core record. Add the SSO configuration for the admin UI, the role assignments, and a dated list of key creations and revocations. Note that SSO is free for up to 5 users in LiteLLM and an Enterprise license is needed beyond that, and that audit logs and organization and team RBAC sit in the Enterprise tier, so confirm which features your records depend on before you buy. The master key should be held in a secret manager and excluded from exports.

Which request logs prove what the assistant processed, and for how long?

Auditors look for logs that record who sent each request, which model handled it, when it happened, and the outcome, along with a written retention period that matches your policy. The gateway access log and the spend records in the database are the primary sources, and the editor server log can confirm that a session used the assistant. Decide whether prompt and response content is logged at all, because storing it can create a new copy of sensitive text and needs its own access controls and retention rule. Set the retention period by policy and confirm with your legal and records team, since requirements for other communications differ by regulation. Log integrity, rotation and off-host copies should be written into the same record.

How do I prove the model endpoint is reachable only from approved hosts?

Produce a network record showing the firewall or security group rules, the listening address of the model runtime, and a test from unapproved hosts that fails. Ollama binds to 127.0.0.1 on port 11434 by default and its local API requires no authentication, so any exposure beyond the host must be intentional and placed behind a proxy or gateway that enforces authentication. Capture the OLLAMA_HOST setting, the reverse proxy configuration, the allowed source ranges, and the results of connection tests from both approved and unapproved segments. Repeat the test after each change. Ask your network team to confirm the approved host list and to keep the scan output with the change tickets.

Who assembles the evidence, and who decides whether it passes?

DeployOpen plans the evidence set, configures the components to produce it, and packages the exports, configuration records and test results into a pack your team can hand to an assessor. The assessment itself is issued by an independent auditor or certification body, and DeployOpen does not issue audit reports, attestations or certifications. Your internal compliance owner decides which records map to which controls in your framework, and the assessor decides whether the evidence is sufficient. Confirm the evidence format, the owner of each record and the review cadence in the written scope before work starts, since those choices determine how much rework the assessment will need.

What would change the evidence list for a regulated business?

The required records depend on the regulation and the data the assistant can reach, so the list should be set by your compliance team before the build starts. Frameworks such as HIPAA require audit controls that record and examine activity in systems containing ePHI, and the Security Rule at 45 CFR 164.312 also covers protection of ePHI in transit. Broker-dealers under SEC Rule 17a-4(b)(4) must preserve copies of business communications for not less than three years, which may apply if assistant output becomes part of a communication. Ask your assessor which controls they will test and which logs must be kept off-host, then map each record in this section to a specific requirement.

What to ask an implementation partner about the AI assistant

Start with the default state, because it decides whether the assistant can reach anything at all. Ask the partner to show, on a test instance, where the Collabora Online 26.04 assistant is enabled or disabled, who can change that setting, and how the change is logged. The assistant ships off by default and administrators choose the AI provider and whether it runs externally or on your own infrastructure, so ask for the exact configuration keys and the change procedure written into the runbook. Then ask which model endpoint they recommend and why. A self-hosted runtime such as Ollama keeps prompts on your hardware, but it does not authenticate local requests on port 11434 by default, so ask how the endpoint is placed behind a gateway that enforces authentication. Ask whether a gateway such as LiteLLM is in scope, and which database and cache it needs for shared limits.

Ask how prompts, responses and gateway activity are logged, where those logs are stored, who can read them and how long they are kept. Ask the partner to name the retention period in writing, to confirm whether prompt content is logged or only metadata, and to show how logs reach your SIEM. Ask how access is restricted by group and key: which groups may use the assistant, how virtual keys are scoped to specific models, and what per-user or per-team budgets and rate limits apply. Ask how the model endpoint is isolated from external networks, including egress rules, whether cloud features are disabled, and how the partner proves that no traffic leaves the boundary during a test.

Ask how Collabora and model updates are tested before they reach production, who signs off, and how rollback works if a new model or gateway release changes output. Ask whether fixed-scope delivery covers initial configuration, testing and documentation, and which items move to a retainer for upgrades and monitoring. Finally, prepare what you must provide: a data classification for the documents the assistant may see, named approvers for the assistant and for log access, and network diagrams showing the editor, document server, gateway and model host. Written answers to these questions form the basis of a scope that can be quoted.

What buyers ask before hiring help for an on-premises AI assistant

What drives the cost of an on-premises AI assistant for document editing?

The cost is set by the components you run and the people who build and support them, not by a single license. The main components are the hardware or cloud capacity for the model endpoint, the LiteLLM gateway and its PostgreSQL database (plus Redis once more than one proxy instance runs), the integration work that connects Collabora Online to the endpoint, logging and review, and the Collabora edition you run. Hardware and infrastructure recur. Integration and testing are usually one-time, with ongoing support after that. LiteLLM is MIT licensed outside its enterprise directory, but SSO beyond five users, SCIM, audit logs and secret manager integrations need an Enterprise license. Ask any quote to list each component, who pays for it and whether it recurs. Collabora's Business plan price is published at 3 euros per user per month for up to 99 users, according to the Collabora pricing page.

How long does an on-premises AI assistant rollout take?

The scope sets the timeline. A pilot with one model endpoint, a gateway and a small user group takes a different path than a rollout across every editing pod, with audit evidence and approval workflows. Timing depends on how many models you evaluate, whether you run the endpoint on GPUs you already own, how long your security and data classification reviews take, and how many hosts and network rules need changes. DeployOpen agrees the duration in writing as part of a fixed-scope engagement before work starts, so ask for the timeline in the scope document rather than a generic estimate.

Can our own team set up an on-premises AI assistant without a partner?

Yes, for a pilot, provided your engineers are comfortable with containers, reverse proxies and API gateways. Ollama and LiteLLM both have public documentation, and Collabora 26.04 includes the assistant as an optional, off-by-default feature. Harder work is the audit side: documenting which models were approved, proving that prompts and outputs are logged and restricted to the right groups, and keeping that evidence current through upgrades. Frameworks such as HIPAA's audit control requirement in 45 CFR 164.312(b) and the record-keeping rules in 17 CFR 240.17a-4 call for evidence you must be able to produce, so teams often bring in outside help for that part.

What does our organization need to provide before work starts?

Several decisions must come from your side before implementation begins. You need named approvers for the AI feature and for the list of models allowed, a data classification that states which documents may be sent to a model endpoint, and an inventory of the hosts that will reach each other. Typical items to confirm include:

  • Which document classes are excluded from AI requests entirely
  • Which groups may use the assistant and which may only view documents
  • The internal hosts for the Collabora Online server, the gateway and the model endpoint
  • Firewall and proxy rules, including whether any outbound model calls are blocked
  • Retention rules for prompt and response logs
Can we keep the AI assistant turned off permanently?

Yes. Collabora Online 26.04 ships the optional AI assistant switched off, and administrators decide whether to enable it and which provider it uses. Keeping it off is a valid long-term choice for teams whose documents must not leave the editing environment. If you keep it off, confirm that the editor configuration, the document server and any connected platform such as Nextcloud do not expose a route to an external AI service. Record the decision and its owner, and review it when Collabora or your platform changes its defaults.

Does DeployOpen certify that our AI assistant is compliant?

No. DeployOpen plans, implements, hardens and documents the system, and prepares the configuration, access controls, logs and evidence that an assessor will review. Independent auditors and certification bodies issue any audit report, attestation or certification. Compliance readiness means your evidence is organized and reproducible before an assessor arrives, which is different from a result an assessor has issued. Ask your auditor early which evidence they expect for AI-assisted editing, since the expectations for logs and model approval records vary by framework and by organization.

Which model endpoint and gateway settings should engineers check first?

Check that the model runtime listens only where you intend. Ollama binds to 127.0.0.1 on port 11434 by default, and its local API does not require authentication, so exposing it through a proxy or changing the OLLAMA_HOST setting must be paired with network restriction and an authenticated gateway in front. Confirm that cloud features are disabled with OLLAMA_NO_CLOUD=1 if you run fully local models. On the gateway, use virtual keys with per-team budgets, model restrictions and rate limits, store LITELLM_MASTER_KEY in a secret manager and keep the database backed up. Pin versions and test each upgrade, because Ollama and LiteLLM both publish frequent releases.

What should we log and review once the assistant is running?

Log who invoked the assistant, which document and model were used, the time of the request, and the decision to allow or block it. Prompt and response content needs a separate decision, because storing it may create a new copy of sensitive text. Review model access regularly, check budget and rate limit breaches, and confirm that revoked users lose access to the gateway. Confirm the exact log fields, retention periods and access controls with your compliance lead, since the required content depends on your sector and the rules that apply to your records.

How DeployOpen scopes and supports an on-premises AI assistant

DeployOpen delivers this kind of assistant as a fixed-scope engagement, with scope, deliverables and timelines agreed in writing before any work starts. For an assistant, the written scope should name each component in the path: the Collabora Online deployment and its AI assistant setting, the model endpoint, the gateway in front of it, the logging sinks, and the user groups allowed to invoke the feature. Each of these is a decision that changes the work, so ask for it to be listed explicitly rather than summarized as "AI enabled." DeployOpen does not publish a price list, and a quote follows from that written scope.

DeployOpen deploys the editor, the model runtime and any gateway inside your own cloud accounts, data centers or air-gapped networks, so prompts and document text stay on infrastructure you control. Hardening covers the items you would expect to review: the feature left off by default, administrator-only enablement, endpoint bindings that are not exposed beyond the required hosts, and access controls on the gateway's virtual keys. DeployOpen also prepares the documentation and the evidence set that your auditors or certification bodies will request, such as configuration baselines, change records and log samples. DeployOpen prepares this evidence but does not issue audit reports, attestations or certifications.

Because an assistant adds a new endpoint that accepts document content, DeployOpen can test it as part of the engagement. That includes security testing of the assistant and its model endpoint, along with AI systems testing that checks prompt handling, data exposure and access restrictions. Testing results feed back into the hardening plan before go-live. After rollout, ongoing upgrades and continuous testing are available under retainers, which matter because editor, model runtime and gateway releases arrive on their own schedules. Confirm in your proposal which retainer covers which component and how a new model or provider change is re-tested.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.