Can a private LLM chat for HIPAA keep ePHI out of vendor model services?
Yes, self-hosting removes a vendor model service from the ePHI path, but it does not take the chat system out of the HIPAA Security Rule. When staff paste a discharge summary or upload a claims file into an assistant, the prompt, the uploaded file, the model's output and the chat history all become electronic protected health information (ePHI) on systems you run. The Security Rule at 45 CFR Part 164 Subpart C applies to covered entities and business associates for that ePHI, so the chat platform, the inference host, the vector store, the database and the backups are all in scope. A self-hosted design changes who holds the data, not whether the controls apply.
Compliance officers should also be clear about what can be shown to an auditor. HHS states that no Security Rule standard requires certification and that it does not endorse or recognize private certifications, so there is no HIPAA certificate to buy for the chat tool. The realistic goal is a set of documented controls, configuration baselines, risk analysis entries and retained evidence that the Office for Civil Rights (OCR) could review in a complaint investigation or compliance review. Ask any vendor or partner who claims HIPAA certification what they mean by it, and what written evidence sits behind the claim.
The practical first decision is the scope boundary. List every component that will hold prompts, files or chat logs, including the interface, the model runtime such as Ollama or vLLM, the database, the file store, the logging stack and any hosting provider or managed service that touches the environment. Each one either needs a business associate agreement that meets 45 CFR 164.314(a) or sits within your own workforce controls. Confirm with counsel and your hosting provider how that analysis applies to your contracts before any real patient data enters a pilot.
Mapping Security Rule standards to chat platform configuration
This table maps the HIPAA Security Rule standards most relevant to a staff chat assistant to the configuration choices in Open WebUI or LibreChat, Redis and PostgreSQL, and the model server. Read each row as a question to answer in your own environment: what the rule asks, how the platform is set up, and what evidence an assessor will expect. The main takeaway is that the rule sets the outcome, while the settings, documents and contracts are what you have to show.
| Requirement | What the function must do | How to configure the chat platform |
|---|---|---|
| 45 CFR 164.312(a)(1), access control | Allow only persons or software granted access rights under 164.308(a)(4) to reach systems holding ePHI, through technical policies and procedures. | Connect SSO through OIDC or LDAP, provision groups with SCIM 2.0, and limit chat, knowledge and model access by role-based groups. Confirm with your vendor how per-resource permissions apply to shared knowledge bases. |
| Unique user identification (required, 164.312(a)(2)(i)) | Assign each user a unique identifier so every chat and file action traces to one person. | Disable shared accounts and generic logins. Map each identity to one account through the identity provider. Restrict API keys to named owners and review them on a schedule. |
| Emergency access procedure (required, 164.312(a)(2)(ii)) | Document how ePHI stays reachable when normal access paths fail, and how access is granted and revoked during an emergency. | Keep a break-glass administrator outside normal SSO, stored under dual control. Test restoration of PostgreSQL and file storage. Confirm which settings change if the identity provider is unavailable. |
| 45 CFR 164.312(b), audit controls | Record and examine activity in systems that contain or use ePHI, including who sent prompts, uploaded files and changed settings. | Retain PostgreSQL audit data and application logs in a central store inside your boundary. Define which events are tracked, who reviews them, and how often. Confirm what chat content the platform logs by default and whether it can be turned off. |
| 45 CFR 164.308(b)(1) and (b)(3), business associate contracts | Obtain satisfactory assurances from any party that creates, receives, maintains or transmits ePHI for you, documented in a written contract or arrangement. | If a host, managed service or support firm can reach the chat data, sign a business associate agreement with it. Confirm whether a model provider or cloud host touches prompts. Keep a subcontractor list, and confirm that each subcontractor has assurances under 164.308(b)(2). |
| Automatic logoff (addressable, 164.312(a)(2)(iii)) | Terminate sessions after a period of inactivity, or document why an equivalent measure is used instead. | Set session lifetimes and idle timeouts in the identity provider and in the chat platform. Confirm how Redis-held sessions expire across replicas. Record the chosen values in your risk analysis. |
| Encryption and decryption (addressable, 164.312(a)(2)(iv)) | Encrypt ePHI at rest where reasonable and appropriate, and document the decision if you do not. | Enable TLS for the browser and internal hops, encrypt PostgreSQL, Redis persistence and shared file storage at rest, and keep model weights and uploads on encrypted volumes. Confirm that key management sits inside your boundary. |
| Network exposure of model servers | Limit who and what can reach the inference endpoint, since it processes prompts that may contain ePHI. | Bind Ollama to an internal address using OLLAMA_HOST instead of the default 127.0.0.1 only if it is needed by other hosts, and place a proxy with authentication in front. For vLLM, bind to an internal interface and restrict access by network policy. Disable cloud features with OLLAMA_NO_CLOUD=1. |
| 164.306(d), documentation of addressable decisions | For each addressable specification, record whether it is reasonable and appropriate, and if not, why, plus any equivalent alternative measure. | Keep a decision record per addressable specification, linked to the HIPAA risk analysis and the configuration baseline. Review it when Open WebUI, LibreChat or Ollama releases change defaults. Ask your partner to deliver this record as part of the scope. |
Which open-source components fit a HIPAA chat deployment
A HIPAA chat deployment is only as strong as the weakest component that touches ePHI, from the interface and identity layer to the model server and the logs. The cards below cover the components most often evaluated for this pattern, what each one contributes to access control, audit and data residency, and what to verify before deployment.
Open WebUI for the chat interface
Open WebUI documents OIDC and LDAP sign-in, SCIM 2.0 user and group provisioning, and role-based access control with groups and per-resource permissions, which map directly to unique user identification and access limits for ePHI. Its license is the constraint to plan around: clause 4 of the Open WebUI License bars altering or removing branding in deployments above 50 end users in any rolling 30-day period unless the operator holds an enterprise license or written permission. Confirm the licensing position for your expected headcount before rollout.
LibreChat as an MIT alternative
LibreChat is an AI chat interface under the MIT license, so the branding restriction that applies to Open WebUI does not apply to it. Compare it against your identity and audit requirements in the same way: confirm which SSO methods, group-based permissions and chat export or deletion controls the chosen version supports, and test them against your access model rather than assuming parity with Open WebUI. Verify the license and the tagged version you deploy.
Ollama bound to localhost
Ollama binds to 127.0.0.1 on port 11434 by default, and its local API does not require authentication, so the gateway in front of it is what protects ePHI. Keep the default bind address unless a reverse proxy with authentication and TLS is placed in front of it, and set OLLAMA_NO_CLOUD=1 or the equivalent server configuration to turn off cloud features so that no prompt is sent to a hosted model service. Verify these settings in your build and document them as part of your configuration baseline.
vLLM for shared GPU serving
vLLM is an Apache-2.0 inference and serving engine suited to a shared internal model service that many staff sessions use at once, where Ollama's per-model concurrency settings may become the limit. It exposes an OpenAI-compatible endpoint that stays inside your network, which keeps model traffic on infrastructure you control. Ask for load testing against your real concurrency, confirm the GPU sizing, and check that the endpoint is reachable only from the chat and gateway tier.
LiteLLM as a logging gateway
LiteLLM sits between the chat interface and the model servers and records requests through its logging features, which gives you one place to capture who sent what to which model. Its enterprise directory uses a separate license, so confirm which features your deployment depends on before choosing a tier. Logs that capture prompts are themselves ePHI, so define where they are stored, who can read them and how long they are kept.
Observability tools and their license limits
Langfuse is MIT licensed outside its ee directories, so a self-hosted deployment can keep traces and evaluations in your own storage, provided you do not use the ee components without an appropriate license. Arize Phoenix uses the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service, so it is suitable only for use inside your own organization and not as a service you sell or outsource. Check each license text against your operating model before adopting either tool.
Where the stack runs and who holds ePHI
When the chat interface, model servers, databases and file storage run in facilities the covered entity owns or leases and operates with its own staff, the hosting question is mostly about who can reach ePHI. Owning the hardware does not by itself create a business associate relationship with a hardware vendor that never sees data. Your own workforce is not a business associate, but an outside firm that can reach prompts, uploads or chat logs generally is one.
- Business associate status: Implementation, support and managed service firms with access to the chat database, object storage or model hosts usually act as business associates. Confirm each firm's access path before work starts.
- Contracts: Execute a business associate agreement with each outside firm that may access ePHI. The written contract requirement in 45 CFR 164.308(b)(3) documents the assurances, and counsel should confirm its scope.
- Controls that stay with you: Physical safeguards under 164.310, including facility access, server room controls and media disposal, remain your responsibility in full.
- Operational load: Patching, backups, log retention and access reviews sit with your team, so staffing must be named before rollout.
Check which outside people hold administrative or remote access, and whether that access reaches the database, the object store or the Ollama or vLLM host that holds prompts.
Running the stack in an organization-owned cloud account means the provider operates the physical facilities and much of the underlying infrastructure, while your team configures identity, networking, storage, encryption and logging. Under HIPAA, a cloud provider that maintains ePHI on your behalf is generally treated as a business associate even if it cannot read the data. Whether the provider's agreement covers the specific services, regions and configurations you use must be confirmed in its current terms, not assumed.
- Provider as business associate: Confirm whether the provider's business associate terms cover each service in your design, such as compute, managed Kubernetes, managed databases and object storage. Services outside those terms should not hold ePHI.
- Contracts: Obtain the provider's business associate agreement where its terms require one, and sign separate agreements with any partner that configures or supports the account.
- Controls that shift: Physical and much of the infrastructure control set moves to the provider. Access rights under 164.312(a)(1), audit controls under 164.312(b), encryption and automatic logoff remain yours to configure and evidence.
- Provider logs and keys: Confirm what provider-native logs record, how long they are kept, who can read them and where encryption keys are held.
Ask the provider which services and regions are covered by its agreement, which are excluded, and how its logging and key management settings can be verified by your team.
In a partner-operated model, an outside firm hosts and runs the chat stack for your organization in its own facilities or a dedicated third-party environment. The partner creates, receives, maintains and transmits ePHI on your behalf, which makes it a business associate. If the partner uses a hosting provider for any part of the stack, that provider is a subcontractor, and the partner must obtain satisfactory assurances from it under 45 CFR 164.308(b)(2) before it handles ePHI.
- Chain of business associates: The partner carries the business associate role, and each downstream hosting provider or support firm needs its own written assurance flowing down the chain.
- Contracts: Sign a business associate agreement with the partner that flows obligations down to subcontractors, and require a current list of those subcontractors with their locations.
- Controls that shift: Facility, workforce and day-to-day operational controls move to the partner. You remain accountable for the risk analysis, access policy and evidence that the controls run, including activity review under 164.312(b).
- Exit terms: Define how chat history, uploads, backups and logs are returned or deleted at the end of service, and how long each copy persists.
Request the subcontractor list, the audit evidence you can review each year, and the location of every copy of chat logs, uploads and backups.
How audit controls must cover chat history and uploads
Section 164.312(b) requires hardware, software or procedural mechanisms that record and examine activity in information systems that contain or use ePHI. For a private chat assistant, the audit scope is the chat history, the uploaded files, the retrieval index and the administrative actions on the platform, not only the login events. NIST SP 800-66 Rev. 2 (section 5.3.2) breaks this into five key activities that map directly to the deployment: decide which activities to track, select the tools for audit and activity review, write the review policy, write standard operating procedures, and run the audit and review process. Start by listing the events the platform can emit, such as sign-ins, role changes, model and provider selection, file uploads, chat creation and deletion, and API key use, then confirm with the vendor and your own testing which of these are logged, where, and with what user identifiers. Open WebUI's administrative and audit features should be verified in your release rather than assumed.
Retention and deletion need the same discipline. Decide how long prompts, responses and uploaded files are kept, who can delete them, and whether users can remove their own chats. Then trace where each file goes: a document added to a knowledge base is usually chunked, embedded and stored in a vector database such as PGVector or Qdrant, so deleting the original upload may not remove its vectors, cached text or backups. Ask your implementation partner to demonstrate a deletion end to end, including the vector store, object storage, database rows and log copies. The retrieval layer also creates a second copy of ePHI. Treat the vector database and any log pipeline as in-scope systems for access control under 164.312(a)(1), for audit under 164.312(b), and for the business associate analysis under 164.308(b), and document them in the risk analysis.
Finally, write the review policy and procedures before go-live. Name the role that reviews audit records, the frequency and the escalation path, and state how reviewers will examine unusual access, such as bulk exports of chat history or repeated retrieval against a single patient record. Keep the logs themselves protected, since audit records that can be altered by the people they monitor provide little assurance. The Security Rule does not certify your program, and HHS does not endorse private certifications, so the evidence you hand an auditor is the policy, the configuration, sample review records and the tested deletion results. Confirm with counsel how long your organization must retain these records under its own policies.
Evidence auditors and OCR reviewers ask for
This table lists the evidence an assessor or OCR reviewer is likely to request for a self-hosted staff chat assistant, the component that produces each item, and what a reviewer tends to look for. Read it as a starting inventory: HHS has no HIPAA certificate, so the strength of the file depends on how well each record matches the environment it describes. OCR's 2024 to 2025 HIPAA audits reviewed 50 covered entities and business associates, so the working files should be ready before a request arrives.
| Evidence | Produced by | What a reviewer checks |
|---|---|---|
| System inventory and data flow diagram | Architecture documentation, maintained with the platform team | Every place chat text, uploads, embeddings and logs are stored, including backups and model hosts |
| Risk analysis and risk management plan | Security and compliance team, with DeployOpen input where scoped | Threats to ePHI in prompts and files, likelihood and impact, and the mitigations chosen |
| Business associate agreements and subcontractor assurances | Legal and procurement, with each hosting or support provider | Written terms under the business associate rules, including any subcontractor handling ePHI |
| SSO and RBAC configuration exports | Identity provider, plus Open WebUI group and permission settings | Only authorized staff can reach chats, models and knowledge collections, matching documented access rights |
| Access review records | Identity provider reports and Open WebUI user lists, tracked in ticketing | Periodic confirmation that accounts, group memberships and admin roles still match current need |
| Audit log samples and review records | Open WebUI admin logs, gateway logs, Ollama and vLLM server logs, shipped to a log store | Logs capture who accessed which chat or file and when, and someone reviews them with a recorded outcome |
| Encryption configuration | TLS termination at the proxy, database and storage settings, key management records | Data in transit and at rest is protected, and the keys are controlled by the organization |
| Backup and emergency access procedures | PostgreSQL backup jobs, object storage snapshots, break-glass account procedure | Backups are encrypted and tested, and a documented route exists to reach ePHI during an outage |
| Documented decisions on addressable specifications | Compliance lead, recorded in the policy repository | For each addressable specification, a reasoned decision and an equivalent measure where one is adopted |
What tooling alone does not satisfy under HIPAA
A self-hosted chat platform can support HIPAA Security Rule controls, but the rule places several duties on the organization and its people. The cards below cover the items that no product configuration can close on its own, and the checks to run before any staff member pastes patient information into the assistant.
No certification exists
There is no HIPAA certification, and HHS does not endorse or recognize private certifications. Any vendor badge that claims HIPAA compliance for a chat platform is not evidence that your deployment meets the Security Rule. Ask vendors for their own documentation of configuration and controls, and treat your own risk analysis and audit records as the evidence an OCR investigator would request.
Risk analysis is organizational work
Software can log events and enforce settings, but it cannot decide which data classes staff may enter, which roles get access to chat history, or how much risk the organization accepts. The Security Rule expects a documented risk analysis and risk management process that covers the assistant, its model host, its vector store and its backups. Assign an owner and repeat the analysis when models, users or data sources change.
Business associate contracts cannot be written by software
If a cloud provider, managed host or support firm can access ePHI in the chat stack, a business associate agreement or equivalent written arrangement under 45 CFR 164.308(b) must be in place before that access begins. Subcontractors that handle the data need their own assurances. Self-hosting inside your own account reduces the number of parties, but it does not remove the need to review every vendor that touches the environment.
Workforce training and sanctions
Staff decide what to paste into the assistant, so the rules for that choice must reach them. Training should cover which record types are allowed, how to use uploaded files, how to report a mistaken disclosure, and what happens when someone ignores the policy. Keep training records and the sanctions policy with the access review evidence, because auditors will ask for both together.
Proposed 2025 Security Rule revision
HHS proposed a major Security Rule revision on January 6, 2025 (90 FR 898), with comments due March 7, 2025. As of October 2026 no final rule has been published, and the latest Unified Agenda lists it under Long-Term Actions with final action projected for July 2027. Design the chat controls against the current rule, and track the proposal so that stronger requirements, such as expanded documentation or asset inventories, can be added without rebuilding the platform.
Default Ollama API has no authentication
The local Ollama API at localhost:11434 does not require authentication, and Ollama binds to 127.0.0.1 by default. Exposing it to other hosts through OLLAMA_HOST or a reverse proxy without network segmentation, TLS and access control would let any reachable client send prompts to the model. Confirm that only the chat application's service account can reach the inference port, and log every request at the proxy.
What do buyers ask before a HIPAA chat rollout?
What does a HIPAA-ready private chat rollout cost, and what drives it?
The cost depends on scope, not on a list price. DeployOpen does not publish a price list, and a quote follows from a written scope. The main cost components are the GPU or server capacity that runs the models (a one-time purchase or a recurring cloud bill), the implementation work (identity integration, hosting, retention and logging configuration, and access controls), the security testing and risk analysis support you request, and ongoing operations. The factors that move the number most are the number of users and peak concurrency, whether models run on your hardware or a cloud account, how many systems the assistant connects to, whether it will ingest files that contain ePHI, and how much evidence you need. Ask any partner to split the quote into one-time and recurring items, and to name who pays for hardware, licenses and support.
How long does a private chat rollout take for a HIPAA environment?
No duration is fixed in advance, so ask for a written scope with a timeline before work starts. Timing depends on how quickly your team can confirm the intended use, approve the data classes allowed in prompts and files, provision the infrastructure, connect your identity provider, and complete the risk analysis that covers the chat history. Pilot phasing, the number of user groups added in waves, and any change control on your side also set the pace. Engagements are typically fixed-scope, which means the scope, deliverables and timeline are agreed in writing before work begins, so you can compare partners on the same written basis.
What does our organization need to provide for the rollout?
Four roles decide whether the rollout moves on schedule. A compliance or privacy owner signs off on the intended use, the allowed data classes and the retention settings. An identity provider administrator configures SSO, groups and provisioning. Data owners for the systems the assistant may read from confirm what ePHI is in scope and who may access it. A named business sign-off approves the pilot and each later user wave. You will also need someone to own the infrastructure account or data center space, and a decision on who receives audit log reviews. Confirm these owners in the written scope so approvals are not a hidden delay.
Can our own IT team build and run a HIPAA chat assistant without a partner?
Some teams can, and the decision should rest on capacity and evidence, not only on technical skill. The open-source components are available under their own licenses, and the work includes GPU sizing, container deployment, PostgreSQL and Redis for multiple replicas, SSO and role mapping, logging, backup and upgrades. Internal teams also need time for a documented risk analysis, the access and audit controls the Security Rule requires, and written contracts with any hosting party that touches ePHI. Map the skills you have against that list, and decide who will patch models and interfaces on a regular cadence. A partner can carry parts of this work under a written scope, while your organization keeps accountability for the compliance program.
Does self-hosting the chat assistant remove our HIPAA obligations?
No. Keeping prompts and models in your own infrastructure reduces the number of third parties that handle ePHI, but it does not remove the Security Rule. Covered entities and business associates must comply with the Security Rule for electronic protected health information under 45 CFR 164.302. The chat history, uploaded files and access logs still need access controls under 45 CFR 164.312(a)(1), audit controls under 45 CFR 164.312(b), and a documented risk analysis. If any hosting, support or cloud party can reach the data, business associate assurances under 45 CFR 164.308(b) and a written contract under 164.314(a) may still apply. Confirm the scope of each party with counsel.
Why document every HIPAA decision if penalties can be so large?
Documentation is how you show that decisions were reasoned and reviewed. The inflation-adjusted HIPAA civil money penalty in 45 CFR 102.3 runs from $145 per violation where there was no knowledge, up to $2,190,294 per violation for uncorrected willful neglect, with a calendar-year cap of $2,190,294, for penalties assessed on or after January 28, 2026. Penalty amounts depend on culpability and the facts of each case, so the practical lesson is to record the risk analysis, the reasons for any addressable specification you did not adopt, the retention settings, the approvals and the review of audit logs. HHS also states that no certification exists for HIPAA compliance, so keep evidence rather than relying on a label.
How DeployOpen scopes a HIPAA-ready chat deployment
DeployOpen plans, implements, hardens and documents the chat stack inside your own infrastructure, whether that is your cloud accounts, your data center or a segregated network. In practice that means the Open WebUI or LibreChat front end, the model runtime, the PostgreSQL database that holds chat history, the shared file storage, identity integration and the logging pipeline are built and configured in your environment, with no prompt or record leaving it. DeployOpen also runs security testing against the application, its APIs and the model endpoints, and prepares the configuration baselines, access matrices and log samples that your risk analysis and your business associate files will reference. What DeployOpen does not do is issue an audit report, an attestation or a HIPAA certification. No such certification exists, and independent assessors remain responsible for any opinion on your compliance.
Most engagements are fixed-scope. Before work starts, the written scope should name the systems in the boundary, the user population and identity sources, the data classes that may be entered (for example, whether ePHI is permitted in prompts or uploads), the retention settings for chats and logs, the test types, the deliverables and the timeline. Ask for the scope to state which items belong to you, such as the signed business associate agreement with any hosting party and the final HIPAA risk analysis, and which items DeployOpen prepares for you to review. Ongoing needs are better handled as a retainer covering version upgrades, patch cycles, periodic security testing and log review support. Because the upstream projects release often, confirm in the agreement how upgrades are tested in a non-production environment before they reach the clinical user base, and who signs off on each change.
When you compare proposals, check three things. First, that the statement of work separates implementation from testing, so the party that configured the audit trail is not the only party that tests it. Second, that the deliverables include the configuration evidence an assessor will ask for, including role mappings, encryption settings, automatic logoff values and log retention. Third, that the retainer defines response expectations for security findings and what happens when an upstream release changes a control. Put these questions to any partner in writing, and expect the answers to be reflected in the final quote.
How DeployOpen delivers HIPAA-ready chat and its evidence
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source chat platforms inside the client's own infrastructure. For HIPAA work, that means the chat stack, its model servers and its logs run where the covered entity or business associate already controls access, and the evidence is prepared for the organization's own risk analysis and any independent assessment it commissions.
How we scope
- Map which workflows may place ePHI in prompts, uploads and chat history before any build starts
- Record which hosting parties, subcontractors and cloud accounts would touch ePHI, so business associate contracts can be checked
- Agree the written scope, deliverables and timeline with the client before work begins, as fixed-scope engagements
- Confirm which NIST SP 800-66 Rev. 2 activities the client expects DeployOpen to document
How we build
- Deploy Open WebUI or LibreChat with models served on the client's own hardware, with no third-party model API in the data path
- Connect SSO and group-based roles so each chat and knowledge base is limited to authorized users
- Configure audit logging for chat access, administrative actions and retrieval events, and test that the records are complete
- Run security testing of the chat application, its APIs and its model endpoints before handover
How we hand over
- Deliver configuration baselines, access matrices and logging settings mapped to the relevant Security Rule standards
- Document the retention, backup and encryption settings, and the procedures the client's staff must follow
- Prepare evidence packages for the client's risk analysis and for any independent auditor it engages, without issuing certifications
- Train administrators on account provisioning, log review and model updates under the agreed procedures
How we support
- Offer ongoing support, upgrades and continuous security testing as a retainer after handover
- Review open-source release notes and test model and platform upgrades before they reach production users
- Repeat access reviews and log reviews on the cadence set in the client's own policies
- Reassess the evidence package when the client changes users, models, hosting or business associate arrangements
