Why financial services need an AI gateway that controls every model call
Many banks and insurers already have staff and applications calling hosted LLM APIs, often without a register entry, a named budget owner or an exit plan. A self-hosted gateway is the control point that closes that gap, because every model call passes through one place where keys are issued, budgets are enforced, approved providers are listed and requests are logged. The first task is an inventory: ask each business unit and development team which endpoints they call, with which keys, and who pays for them. Those answers define the gateway's initial configuration and the first version of the register.
DORA, Regulation (EU) 2022/2554, has applied since 17 January 2025 to 20 types of financial entity, from credit and payment institutions to insurers. Its contract and register rules are where hosted LLM APIs become a governance issue. Article 28(3) requires a register of information covering all contractual arrangements for ICT services, and Article 28(8) requires documented, tested exit strategies for ICT services that support critical or important functions. Article 30(2) requires each ICT services contract to state where services are provided and data processed. A model provider that receives customer data or feeds a credit or claims workflow is therefore best treated as an ICT third-party service, a classification to confirm with your compliance and legal teams for each use. The DORA text on EUR-Lex is the natural starting point for that review.
For the platform, LiteLLM covers keys, budgets and provider routing. Its virtual keys need PostgreSQL and support per-key, per-user and per-team budgets, TPM and RPM limits and restriction to specific models, so each team can call only the models its use case is approved for. Its router offers failover with retries, cooldowns and priority ordering. Redis 7.0 or newer becomes necessary once more than one proxy instance runs. SSO for the admin UI is free for up to 5 users, while audit logs, team RBAC and secret manager integrations sit in the Enterprise license, so confirm that cost before the design is fixed. Where the institution already runs Apache APISIX for API governance, it can handle OpenID Connect authentication and traffic policy at the front door, with LiteLLM behind it for model keys and budgets. Confirm which AI gateway plugins your APISIX version provides before relying on them for guardrails.
Which financial regulations shape the gateway build
This table maps each financial regulation or control framework in scope to what it asks of the gateway and the configuration it drives. Read each row as a design input: the requirement, the gateway setting that answers it, and the evidence an assessor or supervisor will ask to see. Confirm the applicability of each regime with your legal, compliance and assessor teams before you finalize the design.
| Requirement | What it means for gateway configuration | Evidence to keep |
|---|---|---|
| DORA Article 28(3) register of information | Every hosted model provider reached through the gateway is an ICT third-party arrangement to record. Keep the gateway's provider list as the source for the register entries. | Current provider inventory, contract references and the mapping of each provider to the applications that call it |
| DORA Article 28(8) tested exit strategy | Configure failover order and approved self-hosted fallback models behind one OpenAI-compatible endpoint so applications do not change when a provider is removed. | Documented exit plan, routing configuration history and recorded results of failover tests |
| DORA Article 30(2) contract terms | Restrict routing to approved provider regions and endpoints, so the contract's stated locations match where prompts and outputs are processed. | Contract clauses on processing locations and data return, plus the gateway's approved endpoint list and change log |
| PCI DSS v4.0.1 scope | If cardholder data or sensitive authentication data can reach a model prompt, the gateway and the model path may fall in the cardholder data environment. Block card data at the gateway and document the scoping decision. | Scoping statement, data flow diagram, redaction rule configuration and results of periodic checks |
| FTC Safeguards Rule MFA (16 CFR 314.4(c)(5)) | For non-bank lenders in FTC jurisdiction, the admin UI and any staff-facing access to the gateway need multi-factor authentication through the identity provider, unless the Qualified Individual approves an equivalent control in writing. | SSO configuration for the admin UI, MFA enforcement settings and any written exception approval |
| SEC Release 33-8810 access and program-change controls | Treat gateway routing rules, guardrails and budgets as controlled programs. Use role-based access for changes, approval before deployment and a versioned configuration history. | Access review records, change tickets, configuration version history and admin activity logs |
| EU AI Act Article 12(1) automatic logging | Enable automatic logging of each model call with timestamp, model, virtual key and team identifiers so high-risk system events can be reconstructed over the system's lifetime. | Log schema, logging coverage per application, sample records and the logging design document |
| EU AI Act Article 19(1) log retention | Set log retention and export to storage the organization controls for at least six months, unless other law requires longer. Confirm the period with legal counsel. | Retention policy, storage location, access controls on logs and retention job records |
Which data and systems pass through the gateway
Start with the data classes that reach model calls, because they determine every later control. In banking and insurance this usually includes client identifiers, claims files, credit narratives, underwriting and trading notes, and in card businesses cardholder data and, in rare cases, sensitive authentication data. Each class needs a documented handling rule: which models may receive it, whether it must be redacted first, and how long the gateway may keep it. The gateway can only enforce rules that the institution's own data classification already defines, so the first task is to confirm those classes with the data governance owner, not to infer them from the application's form fields.
PCI DSS scope is the point where this exercise becomes a compliance question. The standard applies to entities that store, process or transmit cardholder data, and also to systems that could affect the security of the cardholder data environment. A gateway that sits between a payments application and an external model provider can fall into that second category even if it never sees a card number, because a compromised gateway could alter prompts or responses that feed payment decisions. Ask the assessor, a Qualified Security Assessor or the acquirer depending on the entity type, to confirm scope in writing before the gateway is connected to any payment workflow. The current active version is PCI DSS v4.0.1, as published by the PCI SSC.
The gateway's own logs become records that hold sensitive data. Prompts, completions and metadata written to PostgreSQL or a SIEM can contain the same client details that the model call carried, so log stores need the same access control, encryption and retention rules as the source systems. Connect the identity provider for admin UI and JWT-based access, PostgreSQL as the store for virtual keys and spend records, Redis once more than one proxy instance runs for shared rate limits and router state, a secret manager for LITELLM_MASTER_KEY and provider credentials, and the SIEM for security events. Confirm with the platform team which of these fields are logged in full, which are hashed, and who can read each store.
Which open-source gateways fit a regulated financial estate
Three open-source gateways cover most of the AI gateway work in a bank or insurer, but they differ in what they are built for. The choice depends on whether you need a model-aware gateway with budgets and keys, a general API gateway that already fronts your estate, or a clear split between gateway and trace storage. Check the license tier for each feature your controls depend on before you commit.
LiteLLM for model routing and budgets
LiteLLM is the option built specifically for model access. It exposes more than 100 LLM providers in the OpenAI request format, with routing modes that include failover and cost-based selection, and per-key, per-user and per-team budgets. Virtual keys need a PostgreSQL database, and multiple proxy instances need Redis. Only the enterprise directory is under a separate license, so confirm which features your controls require, because SSO is free for up to 5 users and beyond that needs an Enterprise license. SCIM, audit logs and team-level guardrails are Enterprise features.
Apache APISIX as the front door
APISIX is a general API gateway that also offers AI gateway plugins, released under Apache-2.0. It suits banks that already run a central API layer and want model traffic to pass through the same authentication, rate limiting and logging controls as other services. Its openid-connect plugin works with Microsoft Entra ID, Okta and Keycloak. Choose among traditional, decoupled and standalone deployment modes based on how many clusters must share one control plane, and test the admin API exposure in each.
Kong Gateway as an API standard
Kong Gateway is also licensed under Apache-2.0 and is widely used as a general API gateway. It fits organizations whose platform team already standardizes on Kong for API management and wants AI traffic governed the same way. Before choosing it for model access, confirm which AI routing, token-based limits and provider-level budgeting features are included in the open-source edition and which require Kong's commercial offerings. Do not assume feature parity with a model-specific gateway.
Identity integration at the gateway
Every gateway choice needs a clear answer on who can call which model. Map SSO and token validation to your identity provider, and decide whether the gateway or the application validates user tokens. In LiteLLM, admin UI SSO covers Entra ID, Okta and generic OIDC, while JWT authentication with identity provider tokens is an Enterprise feature. In APISIX, the openid-connect plugin supports authorization code, client credentials and token introspection flows. Ask your partner to document which flow each team uses.
Trace storage kept under your control
The gateway is not the only record. Tracing tools hold prompts, outputs and costs, and their licenses decide what you may run. Langfuse is MIT licensed outside its ee directories, so a self-hosted deployment can run in your own storage under the core license. Arize Phoenix is under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service. Confirm that your internal use and any partner-operated service fit each license before you choose where traces are stored.
Enterprise features to price in
Before the architecture review, list every control your audit requires and mark which open-source features cover it. For LiteLLM, the items likely to need an Enterprise license include SCIM, audit logs, organization and team RBAC, IP allowlisting and secret manager integrations. Ask vendors and your partner for written confirmation of each. A license decision changes the cost model, the support model and the upgrade path, so settle it at design time.
Deployment patterns for banks, insurers and their regulators' expectations
A bank or insurer needs the gateway, its state, its approved endpoints and its logs to sit where the data is allowed to go and where the register of information can point to them. The four components below describe that placement, and each one carries a check that a supervisor or internal auditor is likely to ask about.
Gateway tier on Kubernetes
Run the LiteLLM proxy with one Uvicorn worker per pod and scale by adding pods, keeping the gateway inside the institution's own cloud account or data center. Each pod is a stateless entry point, so capacity changes do not change where prompts flow. Confirm that the admin plane, the Kubernetes cluster and the image registry are all in approved infrastructure and that LITELLM_MASTER_KEY (which must start with sk-) is held in a secret manager, not in manifests.
PostgreSQL as the state store
Virtual keys, per-team budgets and spend records need a PostgreSQL database, which is the primary supported database for the proxy. This store is the evidence base for chargeback and for showing which team consumed which model. Ask where the database runs, who holds backup and restore rights, and how spend records are retained, because they are regulated records in their own right.
Redis for shared limits across instances
Once more than one proxy instance runs, Redis 7.0 or newer holds shared rate limits, router state and cache. Without it, each instance enforces limits on its own, so a team could exceed a budget by spreading calls across pods. Treat the Redis deployment as part of the same control boundary as the gateway, with its own access controls, encryption settings and failover test.
Approved endpoints and routing
Map every permitted model to either a self-hosted endpoint on vLLM or Ollama inside the institution, or a contracted provider that appears in the register of information with its processing location stated in the contract. Restrict each key to named models and use priority ordering so failover never sends regulated prompts to an unapproved provider. Confirm which routing strategies and fallbacks are permitted for each data classification.
Log export and retention
Export gateway request logs to storage the institution controls, with a retention period set by its own policy and by any applicable regulation. Logs should record which model received which data class, the calling team and the key used, so that they can answer a supervisor's question about a specific provider. Confirm how logs are exported, who can alter them, and how a log from a failed or switched provider is kept.
How deployment choices change for on-premises, private cloud and hybrid estates
An on-premises gateway keeps the whole request path inside facilities the institution owns or leases directly. The gateway, its PostgreSQL database and Redis instances, and the hosted or self-hosted model endpoints it calls all sit on networks the institution administers. This gives the strongest control over where prompts travel, but the institution also owns patching, capacity, failover between sites and the secret store.
- Key storage: Keep the
LITELLM_MASTER_KEY(which must start with sk-) and provider API keys in a hardware-backed or Vault-style secret store inside the data center, not in configuration files on the gateway hosts. - Secret manager integration: Integration with HashiCorp Vault or CyberArk is listed among LiteLLM Enterprise features, so confirm with the vendor whether the open-source build can read secrets from a local vault or whether that requires the Enterprise license.
- Network egress controls: Use firewall rules that allow the gateway to reach only the approved provider endpoints, and block all other outbound paths from application subnets.
- Log location: Store prompt and response logs in the institution's own PostgreSQL or log platform in the same data center, with retention set to meet DORA and AI Act record-keeping needs.
Confirm with the vendor and your security team which audit log and IP allowlisting features are available without an Enterprise license before you finalize the design.
In a private cloud account, the institution runs the gateway in its own cloud subscription or tenancy, with infrastructure as code, its own identity controls and its own logging. The provider may be operated by the institution or by a cloud provider under the institution's account governance. Key management and logging integrate with the cloud platform's native services, which reduces some operational work but adds shared-responsibility boundaries that must be documented.
- Key storage: Use the cloud key management service to protect the encryption keys for the database and secrets, and keep provider API keys in the cloud secret manager with access limited to the gateway's workload identity.
- Secret manager integration: Cloud secret manager integrations (AWS, Azure, Google) are named among LiteLLM Enterprise features, so confirm whether the chosen cloud is supported in your license tier.
- Network egress controls: Place the gateway in a private subnet with a managed egress path, and restrict outbound routes to the approved provider endpoints or to private endpoints for self-hosted models.
- Log location: Send gateway logs to storage in an account and region the institution controls, with immutable retention settings and access restricted to audit and security roles.
- Region pinning: Document the regions where prompts, logs and backups are stored, since DORA Article 30(2) requires contracts to state where services are provided and data processed.
Ask your cloud team to confirm which managed services they will operate and which responsibilities remain with the institution.
A hybrid estate sends some traffic to self-hosted models in the institution's infrastructure and some to contracted external model APIs. The gateway becomes the single enforcement point that decides which requests may leave. This model needs the most careful egress design, because every external provider is an ICT third-party service with its own contract, location and exit terms.
- Key storage: Keep each external provider's key in the secret manager with a separate reference per provider, so keys can be rotated or revoked without changing applications.
- Secret manager integration: Confirm whether provider keys can be pulled from an external secret store in your chosen deployment, since the Enterprise feature list includes this capability.
- Network egress controls: Allow outbound traffic only to named provider endpoints, pin the approved regions, and log every outbound call with the provider and model name.
- Log location: Keep one central log of which model saw which request, stored in the institution's infrastructure, even when the provider also keeps its own records. Providers' retention does not replace the institution's own logs.
- Provider register: Record each external endpoint in the register of information, along with the contract terms for data location and data return.
Verify with each provider which regions process your data and which logs they retain before routing any regulated request to them.
How a financial institution rolls out the gateway in phases
The sequence moves from knowing which models are already in use, through decisions on data and endpoints, to a tested gateway that the institution's own teams can operate and exit. Each phase closes with a named output that feeds the next.
Inventory current model use and providers
Your team and the implementation partner list every application, script, vendor tool and employee workflow that calls a model API, including the provider, the account owner and the data types involved. The output is an inventory that maps each provider to a contractual arrangement, ready to feed the register of information under DORA Article 28(3). The step is done when the business owners have signed off that the list is complete, and the gaps they know about are logged as open items.
Classify data and approve endpoints
Risk, compliance and data owners classify the data each use case may send to a model, then decide which providers and self-hosted models are approved for each class. Decisions cover where processing happens, what contract terms apply, and which uses are blocked outright. The output is a written endpoint decision record that the gateway configuration will enforce, and it is complete when every inventoried use case maps to an approved route or a documented exception.
Build the reference gateway in non-production
The implementation partner deploys the chosen gateway, such as LiteLLM with PostgreSQL and Redis or Apache APISIX, into a non-production environment inside the institution's own cloud account or data center. Routing rules, fallback order and model aliases are set according to the endpoint decisions. The output is a reference build with version pinning, infrastructure as code and test results showing that a request to a blocked provider is refused.
Integrate identity and key management
The team connects the admin interface and gateway access to the institution's identity provider, and moves master keys and provider credentials into an approved secret manager. Application access is set up through virtual keys tied to services and teams rather than shared credentials. The step is complete when administrator access requires the institution's standard multi-factor sign-in, and a test shows that a revoked user or key cannot make calls.
Configure budgets and model allowlists per team
Finance and team leads agree spend limits, rate limits and the models each team or application may call. The partner loads these into the gateway and checks that a team reaching its budget is stopped, and that a model outside its allowlist is refused. The output is a signed allocation table that links each key to a cost center, which finance can use for chargeback and which the risk team can review against the inventory.
Set up logging and retention, then test against evidence needs
Every model call is logged with the caller, model, provider, time and cost, and the team sets retention periods and access controls for the log store. Compliance and internal audit list the evidence they will request, such as records of which model saw which data, and run a sample request through to confirm that each item can be produced. The output is an evidence map linking each control to a log field, retention rule and owner, and the step closes when the sample retrieval succeeds.
Test the exit path with one workload
The team selects one low-risk workload and moves it from a hosted provider to a self-hosted or alternative model through the gateway, without changing the calling application. The test records how long the switch takes, which configuration changes are needed, and what breaks. The output is a documented exit test report that the board or risk committee can review as evidence for the exit strategy required by DORA Article 28(8).
Hand over with runbooks and ownership
The partner delivers runbooks for key rotation, provider onboarding, incident response, budget changes, upgrades and restore from backup, and the institution's operations staff run at least one of these procedures themselves while the partner observes. Ownership of each control, the register entry and the approval workflow for new providers is written down. The step is complete when named internal owners accept the handover in writing and open support terms are agreed in a separate retainer or contract.
Which evidence auditors and supervisors ask the gateway to produce
This table maps each supervisory or audit expectation to the gateway component that can generate the evidence, the record an assessor usually asks for, and where the evidence tends to be missing. DeployOpen prepares this evidence inside the client's environment, but auditors, supervisors and certification bodies draw any conclusions about whether the requirement is met.
| Requirement | Gateway component that produces evidence | Evidence to prepare | Where it usually fails |
|---|---|---|---|
| DORA Article 28(3) and Article 30(2): register of information and contract checks | Approved-endpoint configuration and provider inventory of model routes, with the region and data location recorded for each route | Exported model and provider inventory, change history of routes, and a mapping of each provider to its contract and data location | Hosted model endpoints added outside the gateway, or routes changed without a record that the contract was reviewed |
| DORA Article 28(8): tested exit strategy | Routing and failover configuration that can point traffic to a self-hosted model or an alternate provider | Dated exit test records showing traffic moved to the alternate route, with results and the transition plan | Failover configured on paper but never exercised, or fallback models that cannot serve the regulated workload |
| SOX-relevant ITGC areas: access to programs and data, and program changes | Admin UI and Admin API access logs, configuration version history, and change records for routes, keys and budgets | Named administrator list, approval records for configuration changes, and logs of who changed which key, budget or guardrail | Shared administrator accounts, changes made directly in the database, or no link between a change and an approved ticket |
| FTC Safeguards Rule 16 CFR 314.4(c)(5): multi-factor authentication | SSO integration for the admin UI and enforcement of the identity provider's MFA policy for administrators and users | Identity provider MFA policy export, list of accounts that can reach the admin UI, and written approval for any equivalent control | Local admin accounts left enabled, or SSO free-tier limits used in a way that leaves some users outside the identity provider |
| EU AI Act Article 12(1): automatic recording of events | Request logs for every model call, including timestamp, calling application, model, and route taken | Log schema, retention settings and sample exports showing that high-risk system calls are recorded without gaps | Logs sampled rather than complete, or logging disabled for some applications or fallback routes |
| EU AI Act Article 19(1): log retention for at least six months | Log storage in the client's own PostgreSQL or log platform with retention policy set per log type | Retention policy document, storage location, and restore test showing logs remain available for the required period | Retention set by a default that nobody reviewed, or logs rotated before the retention period ends |
| PCI DSS scope for cardholder data environment | Redaction or guardrail rules that stop card numbers from reaching external providers, with logs of blocked requests | Scope statement showing which routes may carry payment data, and test results from guardrail checks | Prompts containing cardholder data stored in logs that are outside the assessed environment |
How much effort, time and input does a bank-grade AI gateway need?
What drives the cost of a self-hosted AI gateway for a bank?
Cost comes from four places: software licenses, infrastructure, integration work and ongoing operations. The LiteLLM core is MIT licensed outside its enterprise directory, so the software itself is not the largest line. The larger items are enterprise features your controls depend on: SSO and SCIM beyond the free tier, audit logs, team RBAC, IP allowlisting and secret manager integrations are listed under the Enterprise license. Infrastructure includes PostgreSQL, which virtual keys and spend tracking require, and Redis 7.0 or newer once more than one proxy instance runs. Integration with your identity provider and SIEM, and the number of approved model providers, add scope. Ask any partner or vendor for a written quote that separates one-time setup from recurring license, hosting and support costs.
Can our own platform team deploy LiteLLM without a partner?
Yes, a capable platform team can install and run LiteLLM alone, and the open-source proxy is documented for production use. The difficulty is in the controls around it. Your team must design key issuance, budgets per team, routing rules, guardrail placement, log retention and high availability, then prove each one to auditors. Teams that already run PostgreSQL, Redis, Kubernetes and an identity provider usually find the base deployment manageable. Teams without those skills, or without time to test failover and restore procedures, often bring in an implementation partner for part of the work. Decide which parts you keep in-house before you request a quote.
What must our institution supply before implementation starts?
Your institution must supply decisions and records that no vendor can make for you. The main inputs are:
- A data classification scheme that says which data may reach which model, including cardholder data in scope for PCI DSS.
- The inventory of current model providers and the contract terms for each, including where services are provided, data processing locations and exit provisions that DORA Article 30(2) requires.
- Identity configuration: the SSO provider, group mappings for teams and roles, and who approves access to the admin UI.
- Logging requirements, retention periods and the SIEM that receives events.
- Named owners for budgets, incident response and change approval.
Without these, the technical build proceeds slowly because every routing and budget rule needs a decision behind it.
How long does an AI gateway rollout take in a regulated bank?
Duration depends on scope, not on the software. A single gateway with a few providers, one identity integration and basic logging takes far less effort than a multi-region deployment with per-team chargeback, DORA exit testing and SIEM integration. The critical path is usually governance: approving providers, agreeing data classes, and documenting the register of information entries and exit plans. Technical tasks like deploying the proxy, configuring virtual keys and testing failover are shorter by comparison. DeployOpen does not publish durations. Its fixed-scope engagements set timelines in writing from the agreed scope, so ask for a phased plan with milestones and the dependencies your own teams must meet.
Does DeployOpen certify our gateway for DORA or PCI DSS?
No. DeployOpen does not issue certifications, attestations or audit reports, and no authority issues a DORA certificate for a gateway. DORA sets no certification scheme for ICT services. For PCI DSS, the assessment is documented as a Report on Compliance by a Qualified Security Assessor or a Self-Assessment Questionnaire with an Attestation of Compliance, and the payment brands decide who must validate. DeployOpen prepares systems, controls and evidence, such as configuration records, access reviews and logging design, so your assessor can review them. Confirm the assessor's requirements before the build starts.
Which LiteLLM features are free, and how long does support last?
From v1.76.0, the LiteLLM admin UI supports SSO with Google, Microsoft Entra ID, Okta and generic OIDC providers at no charge for up to 5 users. Beyond that, an Enterprise license is required. SCIM, audit logs, organization and team RBAC and secret manager integrations are Enterprise features in the vendor's documentation. LiteLLM maintains support for the four most recent stable minor versions for both open-source and Enterprise deployments, so your upgrade plan must fit that window. Confirm current feature boundaries with the vendor before you commit, because licensing terms change.
What ongoing effort does a bank-grade gateway require after go-live?
Ongoing work is steady rather than occasional. Your team must patch the proxy within the supported minor-version window, rotate the master key kept in a secret manager, review budgets and virtual keys as teams change, and test failover between providers. Database operations include backups, restore tests and monitoring of spend-update batching. Logs must be reviewed and retained under your policy. Each change to routing or guardrails should pass change approval and leave evidence. Ask any partner whether ongoing support is a fixed retainer and what it covers, including upgrades, security testing and incident response.
Which questions to put to an implementation partner for a regulated gateway
Ask the partner to show, in writing, how routing and failover are constrained by your approved-provider list. The question is not whether the gateway supports failover, since the LiteLLM router offers retries, cooldowns and priority ordering for failover, but whether a request can ever fall through to a provider that is not on the register of information. Ask for the model group configuration that would be deployed, a test that proves a non-approved endpoint is unreachable from every route, and who approves a change to that list. Ask how virtual keys map to cost centers and teams, and whether a team_id can be issued to an application that the finance function has not registered. LiteLLM virtual keys need a PostgreSQL database and support per-key, per-user and per-team budgets, so ask who owns the budget hierarchy and how temporary budget increases are approved and recorded.
Ask where prompt and response logs are stored, for how long, and who can read them. A partner should name the database, the retention job, the encryption and backup approach, and the role model for the admin UI. Ask which LiteLLM capabilities need an enterprise license: SSO and SCIM beyond the free limit, audit logs, organization and team RBAC, IP allowlisting, and team-level guardrails and logging are listed as Enterprise features, so confirm the current license terms with the vendor before the design is fixed. Ask how guardrails and PII filtering run before any call leaves your network, and request a test set showing blocked and redacted requests. Ask how exit tests are designed and recorded: which routes are switched to a self-hosted model, what evidence is kept, and how results feed the exit strategy that DORA Article 28(8) requires.
Ask how the partner handles upgrades under the four-minor-version support policy LiteLLM describes for open source and Enterprise deployments. The question is how a minor release is tested against your routing rules, guardrails and logging before it reaches production, and how the partner keeps a rollback path. Finally, ask how the partner documents evidence for your own auditors and internal audit: configuration exports, change records, access reviews, test results and log samples, mapped to the DORA articles and internal controls you are assessing. DeployOpen prepares systems and evidence for assessments but does not issue audit reports, so ask the partner to state which artifacts are produced by the engagement and which must be produced by your own team.
How DeployOpen scopes and supports a financial services gateway
The sequence below turns a regulated institution's gateway requirements into a written scope, a gateway running in its own infrastructure, tested controls and an evidence pack that its own auditors can review. DeployOpen does the implementation, testing and evidence preparation, while the institution keeps decision rights over providers, data and risk acceptance.
Agree the scope in writing
Before any work starts, DeployOpen and the institution agree the scope, deliverables and timeline in a written document. For this gateway, that covers the model providers and self-hosted endpoints in scope, the teams and applications that will hold virtual keys, the data classes each endpoint may receive, the retention period for prompt logs, and the identity provider for admin access. The step closes when the institution's risk, compliance and architecture owners have signed the scope.
Map providers and data flows
DeployOpen works with the institution's architecture and third-party risk teams to inventory every model provider and internal model service that the gateway will reach, and to document where each request and response is processed and stored. The output is a provider and data flow register that the institution can reconcile against its own register of ICT third-party arrangements. The step is complete when each endpoint has a named owner and an approved data classification.
Build the gateway in your infrastructure
DeployOpen deploys the chosen gateway, such as LiteLLM or Apache APISIX, with PostgreSQL and Redis inside the institution's cloud accounts, data centers or network segment. Configuration covers routing and failover rules, virtual keys and budgets per team, model allowlists, and connections to the internal identity provider. The gateway runs under the institution's own administrators, and the step closes when a test request passes through every approved route and is blocked on every unapproved one.
Configure guardrails and logging
DeployOpen sets up request logging, retention and access controls for the gateway, and configures guardrails for personal data and unsafe content where the institution's policy requires them. Logs are written to storage the institution controls, with access limited to named roles. The step is complete when a sample of calls can be traced from the calling application to the model endpoint and the response, with the fields the institution's policy requires.
Harden and test the gateway and its endpoints
DeployOpen hardens the gateway host, containers, admin interface and secrets handling, then runs security testing against the gateway's APIs, its admin UI and the AI endpoints it exposes, including prompt injection and data leakage scenarios. Findings are logged with severity, remediation owner and retest status. The step closes when agreed critical and high findings are fixed and retested, and the remaining findings are documented for the institution's risk acceptance.
Prepare the exit path and evidence pack
DeployOpen documents the configuration, change process, access reviews, log samples and test results that the institution will need for internal audit, regulators and external assessors. It also documents the exit path, showing how traffic can be moved from a hosted provider to a self-hosted model through the same gateway endpoint. DeployOpen prepares this evidence but does not issue audit reports, attestations or certifications. Independent auditors and certification bodies make those determinations. The step is complete when the institution's compliance team confirms the pack covers its assessment requirements.
Hand over and support through retainers
DeployOpen hands over runbooks, administrator training and the operating documentation, and the institution's own team takes operational ownership. Where the institution wants continuing help, it can contract a retainer covering upgrades, configuration changes and continuous testing of the gateway and its endpoints. Quotes for ongoing work follow from the written scope, since DeployOpen does not publish a price list. The step closes with a signed handover record and a defined owner for each runbook.
