Why an AI gateway for healthcare starts with PHI routing
A clinical or operations leader will often find that staff already paste patient details into hosted chat tools. A self-hosted AI gateway enforces which models may see protected health information (PHI), and LiteLLM or Apache APISIX are the main open-source candidates to evaluate for that role. The decision that matters first is not which model is best but which models are allowed to receive PHI at all, and the gateway is where that rule can be enforced for every application rather than trusted to individual users.
HIPAA does not offer a certification that an organization can display, so there is no badge to buy. HHS states that no Security Rule standard requires certifying compliance, and private certifications do not prevent a later finding of violation. What the gateway provides is evidence: a record of which applications called which models, which keys and teams were involved, and which requests were blocked. The Security Rule applies to covered entities and to their business associates for all electronic PHI they create, receive, maintain or transmit, so a model provider that handles PHI on your behalf is very likely a business associate. Under 45 CFR 164.314, that relationship must be backed by a business associate contract requiring Security Rule compliance, flow-down to subcontractors, and reporting of security incidents. Confirm with counsel which hosted providers will sign such terms before any PHI route is opened to them.
The Security Rule itself is under revision. HHS proposed a major update on 6 January 2025, and as of October 2026 no final rule has been published, with the Unified Agenda projecting final action for July 2027. Plan for the current rule while designing controls that would map cleanly onto a stricter one, such as documented routing policy, per-key access records and tested log retention. Ask your architecture partner to show how the gateway's configuration is versioned and reviewed, since that history is the evidence an assessor will ask for.
Which HIPAA and related obligations shape the gateway build
This table maps the HIPAA Security Rule safeguards and related duties to what each one means for an AI gateway that may carry PHI. Read each row as a design requirement: the source tells you where the duty comes from, and the last column tells your team what to configure, log or contract for. Appendix A lists 36 implementation specifications, 14 required and 22 addressable, so treat the addressable ones as decisions you must document, not options you can skip.
| Requirement | Source | What it means for the gateway build |
|---|---|---|
| Access control and unique user identification | 45 CFR 164.312 (technical safeguards); unique user identification is a required specification | Give every person and service its own identity at the gateway. Shared admin logins and shared virtual keys for clinical tools will not show who sent which PHI prompt to which model. |
| Audit controls and activity records | 45 CFR 164.312 (audit controls standard) | Log who called which model, when, from which application and with what data class. Confirm where logs are stored, who can read them and how long they are kept under your own retention policy. |
| Encryption in transit and at rest | 45 CFR 164.312 (encryption and decryption specifications are addressable) | Use TLS between applications, the gateway, the database and each model endpoint. Encrypt prompt logs, Redis and PostgreSQL storage, and document how you decided on each addressable encryption specification. |
| Automatic logoff | 45 CFR 164.312 (automatic logoff specification is addressable) | Set session timeouts for the admin UI and developer consoles. Ask your identity provider and the gateway vendor which idle-timeout settings are supported in your version. |
| Business associate contract flow-down to subcontractors | 45 CFR 164.314(a)(2)(i) | Any hosted model provider or managed service that handles ePHI needs a business associate agreement. The gateway register should show which providers have signed one and which must be blocked for PHI routes. |
| Security incident reporting | 45 CFR 164.314(a)(2)(i) requires business associates to report security incidents to the covered entity | Define alerts for blocked PHI routing attempts, failed guardrail checks and unexpected provider calls. Agree on the reporting path and timing in the contract and the runbook before go-live. |
| Minimum necessary use of PHI | HIPAA Privacy Rule (45 CFR Part 164 Subpart E), summarized here for design purposes | Redact or tokenize identifiers before any external call. Use guardrail rules to strip names, record numbers and dates where the task does not need them. Confirm detection accuracy with your own test set. |
| EU-facing duties where relevant | GDPR Articles 32 and 34; NIS2 Annex I health sector | If EU patients or EU-based providers are involved, route PHI only to approved EU endpoints and keep logs that support breach notification. Confirm with counsel whether your organization falls under NIS2 scope. |
Local, contracted and external model routes for PHI
Models served with vLLM or Ollama inside the hospital network are the simplest route for PHI because prompts, completions and logs stay under the organization's own control. The gateway still has to enforce that a request tagged as containing ePHI can reach only these endpoints, and that the endpoints themselves are covered by the same access and audit controls as other clinical systems. Under 45 CFR 164.312, unique user identification, automatic logoff and encryption are the technical safeguards most often examined for a shared inference service.
- Allowed data: Any ePHI the care or administrative use case requires, subject to the minimum necessary and the organization's own policies.
- Gateway checks: Model allowlist by data classification, unique virtual keys per application and user group, and TLS between the gateway and the inference servers.
- Logging: Request metadata, model name, caller identity and timestamp written to storage the organization controls, with a defined retention period.
- Failover: Routing only among local replicas, so a local outage degrades the service instead of silently sending PHI elsewhere.
- Confirm internally: Who administers the GPU hosts, where model weights and caches are stored, and whether any telemetry leaves the network.
Before go-live, confirm that the self-hosted route is covered by your risk analysis and that the inference hosts appear in the asset inventory.
Some clinical or administrative workloads may need a model hosted by a contracted provider. For PHI, that route is only usable when the provider is a business associate under a written agreement that meets 45 CFR 164.314(a)(2)(i), and when traffic reaches the provider over a private endpoint instead of the public internet. The gateway should treat this route as a named, separately approved destination, not as a general provider option.
- Allowed data: Only the data categories named in the business associate agreement and the approved use case.
- Gateway checks: Route bound to a private endpoint address, provider registered in the approved-provider list, and PHI-tagged requests blocked from any other provider.
- Contract terms to confirm: Security Rule compliance, flow-down of the same terms to subcontractors that handle ePHI, and the incident reporting obligation to your organization.
- Data retention to confirm: Whether prompts and completions are stored, for how long, in which region, whether they train models, and how deletion is evidenced.
- Operational terms to confirm: Subprocessors, breach notification timing, and how the provider supports access logs you need for audit.
Do not enable this route for PHI until the signed agreement and the provider's retention statements have been reviewed by legal and privacy staff.
Public hosted model APIs without a business associate agreement are not an approved destination for PHI. The control here is a hard block, enforced at the gateway, that does not depend on developers choosing the right model name. Blocking must work even when an application sends a request with a model string that the gateway has never seen, so the default for PHI-tagged traffic is deny.
- Blocked by default: Every external provider endpoint for requests carrying a PHI tag or a sensitive data label.
- Gateway checks: Deny-by-default provider policy, rejection of requests that name an unlisted model, and alerts when a blocked request is attempted.
- Redaction path: Where a use case must reach an external model, the gateway should redact identifiers before the call and log the redaction, which is a different control from routing and needs its own validation.
- Logs: Blocked attempts recorded with caller, requested model, data tag and decision, so the security team can review patterns.
- Confirm with the vendor: If an external provider is ever proposed, confirm the agreement type, retention and training terms before any policy change.
Test the block with a deliberately tagged request in a non-production environment and keep the result as evidence.
Which gateway and guardrail platforms fit a health system
A health system needs one controlled path for model calls, guardrails that inspect what goes in and comes out, and traces that clinical governance and privacy teams can review. The cards below cover the components that usually sit in that path and the licensing and operating questions to settle before any PHI reaches them.
LiteLLM as the gateway core
LiteLLM sits between applications and model providers and handles virtual keys, per-team budgets, model allowlists and routing. It is MIT licensed outside its enterprise directory, but virtual keys depend on PostgreSQL, and running more than one proxy instance requires Redis 7.0 or newer for shared rate limits and router state. SSO is free for up to 5 users, so confirm early whether clinician-wide SSO, SCIM provisioning, audit logs and RBAC need an Enterprise license. Those features matter for HIPAA access control and audit evidence.
Apache APISIX as the edge gateway
Apache APISIX is an Apache-2.0 API gateway with AI gateway plugins. It fits a health system that already runs APISIX for clinical or partner APIs and wants model traffic handled by the same edge controls for authentication, rate limiting and observability. Confirm which AI plugins your version provides for provider allowlisting and request inspection, and whether they cover the per-key budget and PHI routing rules you need. Where APISIX is the edge and LiteLLM is the policy layer, decide which one owns budgets and logs.
NeMo Guardrails for input and output rails
NeMo Guardrails is an Apache 2.0 library that adds input rails, output rails, retrieval rails and execution rails to a model call. For PHI, input rails can detect identifiers and jailbreak attempts before a prompt leaves the gateway, and output rails can check responses for leaked identifiers. Colang 1.0 is the default configuration language, and migrating to Colang 2.x needs manual review after conversion. Test the rails against your own clinical note samples, because detection accuracy for medical identifiers is not established by the library alone.
Langfuse for trace review
Langfuse is an MIT-licensed LLM observability platform, with its ee directories under separate terms, so traces of prompts, outputs and costs can stay in your own storage. Clinical governance and privacy teams need to review which model saw which record, so store traces with PHI handling rules that match your retention policy. Phoenix is an alternative, but its Elastic License 2.0 prohibits offering the software to third parties as a hosted or managed service, so confirm that the way your partner operates it is permitted.
Open WebUI or LibreChat for clinicians
Open WebUI or LibreChat gives clinicians a chat front end that calls the gateway rather than providers directly. Open WebUI's license restricts altering or removing its branding except for small deployments (no more than 50 end users in any rolling 30 day period), written permission or other listed exceptions, so a health system with a larger user base needs to confirm branding terms before rollout. LibreChat is MIT licensed. Either way, confirm that chat history storage, file uploads and session timeouts meet your HIPAA safeguards for automatic logoff.
Which clinical and administrative systems send data to the gateway
The first decision is which systems will call the gateway, because each one creates a different PHI exposure. EHR-embedded assistants that summarize a chart or draft a reply send the most sensitive material, since the prompt usually carries the patient identifier, the problem list and recent notes. Clinical documentation tools send dictation transcripts and visit notes. Coding and revenue cycle helpers send discharge summaries, claim narratives and diagnosis lists that map to billing codes. Patient messaging drafts send the patient's own question along with the record context needed to answer it. Research notes may look de-identified but often keep dates, rare conditions or ward names that still identify people. Build an inventory with one row per system, listing the calling application, the virtual key it uses, the model routes it may reach and the data classes it can send. Confirm each row with the application owner before any route is opened.
PHI rarely sits only in structured fields. Identifiers such as names, medical record numbers, dates of birth and addresses appear in the message body, in free-text clinical notes and inside attachments such as scanned letters, PDFs and images, where text extraction can expose data a form field would never show. A gateway that only checks the JSON metadata will miss most of it. Redaction therefore has to run inside the gateway, through a guardrail or pre-call callback, before any request leaves for a model outside your boundary. Test this with synthetic records that include the identifier types your sources actually produce, and confirm whether your chosen guardrail library covers attachments or only text. Log the redaction result, not the original value, so the audit trail does not become a second copy of the PHI.
Identity has to tie each call to a clinician, coder or staff member, because HIPAA access controls depend on unique user identification. Common candidates are the hospital directory via LDAP, Microsoft Entra ID, Okta or a generic OIDC provider. Confirm with the vendor and your own test tenant which of these the LiteLLM admin UI accepts for single sign-on in your version, and whether group claims can drive team membership and model access. Note that the LiteLLM documentation ties SSO beyond five users to an Enterprise license, and that SCIM provisioning and JWT authentication with identity provider tokens sit in the Enterprise tier. Where the application passes its own user token, check that the gateway validates it rather than trusting a forwarded header. Document the mapping from directory groups to teams and keys before go-live.
How a health system rolls out the gateway in phases
The sequence moves from knowing every model API in use to a governed, evidenced gateway that staff trust with PHI-adjacent workloads, with each phase closing on a named output.
Inventory every model API in use
Start by finding every place a model is called: sanctioned chat tools, clinical and administrative applications, vendor features that call hosted models, and developer experiments with API keys. Assign an owner to each application, record the provider, the endpoint and the data it receives. DeployOpen typically runs this with the security team and application owners, and it is done when the inventory is reviewed and every entry has an owner and a provider. Confirm with each vendor which of their features send prompts to third-party models.
Classify data and set the PHI policy
Define which data classes count as PHI or ePHI for AI use, which may go only to self-hosted or contracted models, and which may never leave the organization. Write the rules as routing and blocking criteria the gateway can enforce, such as approved provider lists per data class and redaction requirements before external calls. The output is a signed policy that privacy, compliance and clinical leadership accept, with a named owner for exceptions.
Design architecture and trust boundaries
Map where the gateway runs, which PostgreSQL and Redis instances hold keys, budgets and state, where prompt logs are stored, and which network paths reach local models and contracted providers. Document identity for the admin UI and for applications, and show which boundaries carry ePHI. The design is complete when the security architect and the compliance lead can trace every model call from application to provider and to the log store.
Build in a non-production environment
Deploy LiteLLM or Apache APISIX in a non-production environment, with NeMo Guardrails configured for input and output rails that match the PHI policy. Configure virtual keys, model allowlists and routing rules in version-controlled configuration. Confirm which LiteLLM features, such as SSO beyond the free user limit, audit logs and team-level guardrails, need an enterprise license before the design is locked. The output is a working build that the team can rebuild from code.
Validate with synthetic PHI-like test data
Test routing, blocking and redaction using synthetic records that mimic PHI patterns, never real patient data. Check that requests with restricted data classes reach only approved endpoints, that unapproved providers are refused, that redaction runs before external calls, and that every call produces a log entry with the fields the audit plan requires. Record results as test evidence with pass or fail outcomes and fixes applied. Validation closes when all policy test cases pass or have accepted exceptions.
Run a controlled pilot with one department
Select one department with a clear use case and a named clinical or operational sponsor. Limit it to the approved models and budgets, train users on what the gateway blocks and why, and monitor logs, false positives in guardrails and latency. Collect feedback and adjust policy and rails through the same change process used in testing. The pilot is complete when the sponsor and the compliance lead agree the controls work in daily use and the open issues are documented.
Harden for production
Move LITELLM_MASTER_KEY, provider credentials and database secrets into a secret manager. Run one Uvicorn worker per pod on Kubernetes and scale horizontally. When more than one proxy instance runs, use Redis 7.0 or newer for shared rate limits, router state and cache, and confirm the PostgreSQL backup and restore process. Set spend update batching as the vendor recommends at scale, and test failover between providers. The step closes when a load and failure test passes and the security team signs off on the configuration.
Hand over runbooks and evidence packs
Deliver runbooks for key rotation, adding a model, responding to a blocked-request spike, log retrieval for audit and incident handling, along with the evidence pack: policy, inventory, architecture, test results, configuration baselines and access reviews. Agree who operates the gateway after handover and how changes are approved. Note that the evidence supports HIPAA Security Rule reviews, but HIPAA has no certification, so auditors and HHS assess the organization's own controls and records.
Evidence the compliance team asks for and which component produces it
This table maps the records a compliance team typically asks for under the HIPAA Security Rule to the gateway component that produces each one and the source it should be checked against. Read it as a starting inventory for an assessment, not as proof of compliance: DeployOpen prepares these records, while independent auditors or certification bodies issue any opinion, and HIPAA has no certification.
| Evidence | Source | Producing component |
|---|---|---|
| Inventory of model providers and business associate agreements on file | Contracts file and gateway model configuration, checked against 45 CFR 164.314 business associate terms | Model registry in gateway config, plus contract records held by the compliance team |
| Per-key and per-team access records | Virtual key and team assignments, with model restrictions and budgets | LiteLLM virtual keys backed by PostgreSQL |
| Unique user identification mapping | Identity provider user IDs linked to gateway users and keys, relevant to the 164.312 unique user identification specification | SSO integration for the admin UI, plus JWT authentication on Enterprise tiers (confirm with the vendor) |
| Request and response logs with model and route | Gateway request log entries recording caller, model, route and timestamp, with retention settings | Gateway logging callbacks shipped to self-hosted storage such as Langfuse or a SIEM |
| Guardrail decisions and block events | Input and output rail results, including redaction and blocked provider calls | NeMo Guardrails configuration enforced at the gateway, with decisions written to the log store |
| Change records for routing and policy | Version-controlled config files, change tickets and approval records for routing rules and rail changes | Git history and deployment pipeline, with gateway config treated as controlled code |
| Encryption configuration | TLS settings on gateway listeners, database and Redis connections, and secret store configuration | Gateway and infrastructure configuration, with keys held in a secret manager (confirm supported integrations for your tier) |
| Incident records for misdirected PHI | Investigation notes, affected records and notifications to the covered entity under business associate terms | Incident response process, using gateway logs as the primary investigation source |
Questions health IT leaders ask before buying an AI gateway
Does HIPAA certify an AI gateway?
No. There is no HIPAA certification for software or for organizations. HHS states that no Security Rule standard requires an entity to certify compliance, and that private certifications do not stop HHS from later finding a violation. The gateway is a tool that supports your own HIPAA risk analysis and safeguards under 45 CFR 164.312, including unique user identification, access controls and audit logging. Your compliance evidence comes from your policies, configuration records and logs, and any independent assessment you choose to commission. Ask any vendor or partner who claims a HIPAA certification for the gateway to explain what that claim means, because the HHS position is that it does not exist. The HHS FAQ on certification explains the position in HHS's own words.
What drives the cost of a self-hosted AI gateway for healthcare?
Cost falls into four groups: infrastructure, data services, licensing and services. Infrastructure covers the compute that runs the gateway and any guardrail services in your cloud account or data center. Data services cover PostgreSQL for virtual keys and spend records, and Redis 7.0 or newer once more than one proxy instance runs for shared rate limits and router state, as the LiteLLM production guide describes. Licensing matters because LiteLLM's SSO and SCIM, audit logs, team RBAC and secret manager integrations sit in its enterprise license, and SSO is free only up to 5 users. Services cover guardrail tuning, PHI detection validation, identity integration, logging design and operations. Ask any quote to separate one-time build work from recurring support, upgrades and licenses, and to name which LiteLLM features it assumes need an enterprise license.
How long does it take to implement an AI gateway for PHI routing?
The timeline depends on scope, and the agreed timeline is written into the fixed-scope engagement before work starts. The main variables are how many model providers and self-hosted models must be onboarded, how many teams need virtual keys and budgets, how complex the PHI classification and redaction rules are, whether SSO and audit log export are required, and how much clinical and privacy review the rules need before go-live. Validation work, such as testing that redaction and routing rules block unapproved providers, usually takes longer than the configuration itself. Ask a partner for a phased plan with named milestones, the sign-off each milestone needs, and which items depend on your staff being available.
What does our team need to provide for the engagement?
Your team supplies the decisions and access that only your organization can give. Typically this means a complete inventory of model providers and self-hosted models in use, including shadow AI tools staff may already use, a data classification scheme that says which fields are PHI, and clinical and privacy sign-off on routing and redaction rules. You also need identity integration: the SSO provider, group mappings for clinical and administrative roles, and an owner for access reviews. Infrastructure access in your cloud account or data center, a named security contact for the risk analysis, and a retention decision for prompt logs and access logs are also required. Confirm these items in the written scope so that gaps do not surface during testing.
Can our internal team build and run this gateway without a partner?
Yes, if you already have platform engineering skills in Kubernetes or container hosting, PostgreSQL, Redis, identity integration and security logging. The open-source LiteLLM proxy and its routing, virtual keys and budgets are approachable for experienced platform teams. The hardest parts are usually the enterprise features your HIPAA controls depend on, such as SSO, audit logs and team-level guardrails, and PHI validation, which requires testing real-looking clinical text against detection and redaction rules and measuring what leaks. Also plan for ongoing upgrades and reviews of guardrail behavior. Ask your own team to estimate effort for those two areas before deciding, and compare that estimate with a written partner scope.
Does the proposed HIPAA Security Rule revision change this decision?
Not for the current decision, but it is worth tracking. HHS proposed a major Security Rule revision on 6 January 2025 (90 FR 898). As of October 2026, no final rule has been published, and the Unified Agenda projects final action for July 2027 under RIN 0945-AA22. Until a final rule is published, the existing Security Rule in 45 CFR Part 164 Subpart C governs your obligations. Design the gateway's logging, access control and encryption settings to meet current requirements and keep them documented, so that a later rule can be assessed against the same records. Ask a partner how they track proposed changes and what a rule change would require in your configuration. You can follow the Unified Agenda entry for RIN 0945-AA22 for the schedule.
What to ask an implementation partner before signing
Start with how PHI classification and redaction are tested. Ask the partner to show a test set of synthetic records covering names, dates of birth, medical record numbers and free-text clinical notes, and to report how each category was detected before and after redaction. Ask which redaction method runs before any external call, where the rule definitions live, and who signs off on changes to them. Then ask how the team validates guardrails against false negatives: a false negative is a PHI leak that the filter did not catch, so the partner should describe the adversarial samples used, the acceptance threshold agreed with your privacy office, and how new misses feed back into the test set. NeMo Guardrails can be wired in as an input or output control, but the partner should confirm in writing which rails run on which routes and how a failed check is logged. Ask for these results before signing, not after go-live.
Next, ask which LiteLLM features need an enterprise license and why. The LiteLLM documentation places SSO beyond the free limit of 5 users, SCIM, audit logs, team-level guardrails and logging, and secret manager integrations under the Enterprise license, and the license terms in the enterprise directory differ from the MIT license covering the rest of the code. Ask the partner to map each control your HIPAA audit plan depends on to a license tier, and to confirm the cost and renewal terms in the written scope. Also ask where audit logs are retained, for how long, who can read them, and whether the partner can show that they are kept in your storage and survive a gateway upgrade. Ask whether a business associate agreement covers the partner's access to these logs, and whether each external model provider has signed one with you before any PHI route is enabled. Confirm which checks the partner runs on those contracts under 45 CFR 164.314, and which party holds the signed copies.
Finally, ask how the gateway runs in high availability with PostgreSQL and Redis, and how keys map to clinical teams. LiteLLM names PostgreSQL as the primary supported database and requires Redis 7.0 or newer once more than one proxy instance runs, so ask for the failover design, backup and restore tests, and what happens to rate limits and virtual key state during a Redis outage. For access, ask how each clinical service and team receives a virtual key restricted to approved models, how the LITELLM_MASTER_KEY is held in a secret manager, and how staff sign in to the admin UI. Last, ask how upgrades are handled. LiteLLM supports the four most recent stable minor versions, so ask for a dated upgrade calendar, a staging test plan that re-runs redaction and guardrail tests after each change, and who approves each release into production.
Integration points that connect the gateway to hospital systems
An AI gateway handling PHI sits between clinical applications, identity, logging and model hosts, so each connection becomes part of the HIPAA safeguard story. The cards below cover the six integration points a hospital team should map and confirm before any prompt containing patient data passes through the gateway.
Identity provider integration
The LiteLLM admin UI supports SSO with Google, Microsoft Entra ID, Okta and generic OIDC providers, and SSO beyond the free limit of 5 users requires an Enterprise license. Confirm which identity provider your hospital uses for clinicians and administrators, whether SCIM deprovisioning is required, and how JWT tokens from your provider map to teams and model allowlists. Unique user identification under the technical safeguards depends on this mapping being exact.
EHR and clinical app connectors
Applications that send prompts to the gateway, such as an EHR add-on, a clinical documentation tool or a patient messaging service, usually authenticate with virtual keys rather than people. Confirm which applications call the gateway, which patient identifiers they forward in prompts, and whether an integration engine or FHIR interface sits between the EHR and the gateway. Each application should hold its own virtual key with a model allowlist and budget.
SIEM and log forwarding
Gateway request logs must reach the hospital security operations center and be retained under the organization's own policy. Confirm which events are forwarded (authentication, key creation, model calls, guardrail blocks), the format, the transport and who reviews them. Audit log features are part of the Enterprise scope, so confirm licensing before designing the forwarding path. Logs that capture prompts may themselves contain PHI and need the same access controls.
Secret manager for master key and provider credentials
LITELLM_MASTER_KEY must start with sk- and should live in a secret manager rather than a configuration file, as LiteLLM's production guidance recommends. Provider API keys for hosted models need the same treatment. Confirm which secret manager the hospital already operates, who can read the master key, how rotation is performed without downtime, and whether the gateway is configured to pull secrets at startup. LiteLLM Enterprise lists integrations with several vaults, which should be verified against your platform.
Network controls and private endpoints
The gateway should accept connections only from approved clinical networks and reach only approved model endpoints. Confirm IP allowlisting support in your license tier, private endpoint options for any hosted provider your policy permits, egress rules that block unapproved provider domains, and whether traffic crosses the public internet at any point. Gaps here are where prompts can leave the network without a log entry, so test egress from the gateway host explicitly.
Self-hosted inference hosts
Requests classified as PHI should route to models running on hospital-controlled hosts, such as vLLM or Ollama servers in the data center. Confirm which hosts serve which models, how the gateway health-checks them, what happens to routing when a host fails, and whether any fallback to a hosted provider is permitted for PHI at all. Routing tables need a documented owner, since a misconfigured fallback is a disclosure path.
How DeployOpen scopes and supports a healthcare AI gateway
DeployOpen is a Chainscore Labs company that plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own cloud accounts, data centers or air-gapped networks. For a healthcare AI gateway, that means the gateway, its PostgreSQL and Redis dependencies, the guardrail layer and the logging pipeline all run in infrastructure the health organization controls. Engagements are fixed-scope, and the scope document is agreed in writing before work starts. For this topic, that document should name the provider allowlist, the PHI classification rules, the redaction points, the admin SSO design and the log retention and access model, so each item can be tested and accepted individually.
Security testing covers the gateway as a system and the controls around it. That includes the virtual key model and admin roles, the routing rules that keep PHI-tagged requests away from unapproved endpoints, and the guardrail behavior when a prompt tries to bypass redaction through injection or encoding tricks. DeployOpen also tests the logging path, checking whether every model call produces a record with the calling identity, model and outcome, and whether those records can be altered by the people they describe. Health organizations should ask for test evidence tied to named scenarios, such as a clinician prompt containing a patient identifier, rather than a general statement that the gateway was tested.
DeployOpen prepares systems, configurations and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications, and there is no HIPAA certification to obtain. Work products typically include configuration baselines, access reviews, log samples and a control-to-evidence map that a covered entity or its auditor can read. Ongoing support, upgrades and continuous testing are available as retainers, which matter because LiteLLM and its guardrail components change between releases and the Security Rule revision proposed in 2025 may still change obligations. Ask any partner how upgrades are regression-tested against the redaction and logging controls before each rollout.
