Portkey vs Kong AI Gateway: which to choose

Portkey vs Kong AI Gateway: which fits a team that already runs Kong for APIs

If your team already runs Kong for APIs and only needs basic LLM proxying, the Apache-2.0 Kong Gateway with the AI Proxy plugin is usually the first option to test, because it adds LLM routing to a gateway your platform team already operates. If you need multi-model load balancing or semantic routing, the decision moves to Kong AI Gateway Enterprise, or to a Portkey deployment that provides those controls out of the box. The choice turns on three questions: whether advanced load balancing and semantic routing are hard requirements, whether Kong Enterprise is already licensed in your estate, and whether you want LLM-specific features such as prompt management available without building them yourself.

The open-source route fits teams that want one control plane for APIs and model traffic and can accept the operating work of Kong's own configuration model. The AI Proxy plugin needs Kong Gateway 3.6 or later and covers the main provider families, including Ollama and vLLM for self-hosted models, so a team with local inference can start without a commercial contract. Confirm the exact plugin version and provider list against the Kong AI Proxy plugin documentation before you commit, since some features depend on later releases.

Portkey fits better when the LLM workload is the main concern and you want gateway, logging and prompt tooling as one product. Its hybrid model keeps the data plane in your VPC, but the control plane stays with Portkey, so review that split against your residency rules before you choose it. Ask Portkey and Kong, in writing, which logging, budget and access-control features sit in each tier, because the gap between Kong's open-source plugins and its enterprise tier is where most of the cost difference will appear.

Which option fits which scenario

This table matches common team situations to the Kong AI Gateway tier that fits best, and to Portkey where Kong's tiers fall short. Read each row against the split between the Apache-2.0 AI Proxy plugin and AI Proxy Advanced, which needs Kong AI Gateway Enterprise. The main takeaway is that basic routing can stay on the open-source gateway, while weighted, latency-based or semantic routing is where licensing decides the answer.

Scenario	Better fit	Reason
Already running Kong, need only basic model routing	Kong Gateway with AI Proxy	AI Proxy is in the Apache-2.0 gateway and covers chat, embeddings and many providers, so the existing Kong estate can carry the traffic.
Need weighted multi-model balancing	Kong AI Proxy Advanced or Portkey	Weighted and priority balancing across models sits in AI Proxy Advanced, which is Enterprise-only. Confirm the licence terms before relying on it.
Need semantic routing by prompt similarity	Kong AI Proxy Advanced or Portkey	Semantic routing is documented only in AI Proxy Advanced. Without an Enterprise license, plan for a different routing layer.
LLM-first team with no API gateway	Portkey gateway or LiteLLM	Running Kong for LLM traffic alone adds an API platform you do not otherwise need. Portkey's gateway or LiteLLM covers routing without that overhead.
Strict data residency	Kong Gateway self-hosted	Kong Gateway runs in your own infrastructure in DB-less, hybrid or PostgreSQL-backed modes. Confirm that Konnect is not part of the data path.
No Kong Enterprise budget	Kong AI Proxy, with Portkey or LiteLLM for advanced routing	Keep basic routing on the open-source plugin, and move only the features that need AI Proxy Advanced to a gateway with a permissive license. Confirm each vendor's current terms.
Need hybrid control plane with vendor-managed logs	Portkey hybrid deployment	Portkey keeps its control plane outside your VPC, which fits teams that accept outbound HTTPS to that service. Confirm the data terms with Portkey.

Which Portkey features each Kong tier replaces

The Apache-2.0 Kong Gateway includes the AI Proxy plugin, which sends application requests to LLM providers through Kong routes. For teams that already run Kong for APIs, it is the lowest-cost starting point for single-model proxying under the same gateway and operating model.

Covered: proxying to providers such as OpenAI, Azure OpenAI, Amazon Bedrock, Anthropic, Gemini, Vertex AI, Cohere, Mistral, Hugging Face, Ollama and vLLM, for the request types listed in the plugin documentation, including chat completions and embeddings.
Not covered: multi-model load balancing and semantic routing, which belong to AI Proxy Advanced in the enterprise offering.
Version floor: AI Proxy needs Kong Gateway 3.6 or later, and some features need 3.10 or 3.11, so match the gateway release to the features in your design.
Built or bought elsewhere: LLM tracing, prompt versioning and team-level budgets are not described as plugin functions in the material reviewed here. Plan a separate tracing store and prompt store, and confirm what Kong's logging plugins can export.

Confirm per-request behavior, supported request types and version requirements against the Kong AI Proxy documentation before you design routes.

Where requests, keys and logs live under each model

Portkey and Kong AI Gateway put different parts of the request path in different places, and that split decides where prompts, provider keys and logs end up. The cards below separate the self-hosted Kong deployment modes, Kong Konnect, and Portkey's hosted and hybrid models, then cover the identity and compliance questions each one raises.

Self-hosted Kong: DB-less or PostgreSQL

Kong Gateway can run DB-less with configuration loaded from declarative files, or backed by PostgreSQL for stored configuration. In either mode, request traffic, provider credentials referenced by the AI Proxy plugin, and any logs you configure stay on hosts you operate. Check where the gateway's logs are written and which plugins forward payloads to external services, since the gateway itself does not decide where prompts go once a plugin is configured.

Self-hosted Kong: hybrid control and data planes

Hybrid mode separates a control plane that holds configuration from data planes that process traffic. Your team operates both, so the data plane can sit close to model endpoints in your network while configuration stays under your administration. Confirm which components hold provider API keys, which need to reach the control plane, and how certificates between the two planes are issued and rotated.

Kong Konnect as managed SaaS

Konnect is Kong's managed, subscription-based platform for analytics, service catalogs and developer portals. Using it for gateway management means configuration and telemetry are handled by Kong's cloud service rather than your own control plane. Ask Kong where Konnect stores data, which regions are available, and whether provider keys or prompt content ever reach Konnect, because the answer changes your data residency review.

Portkey hosted and hybrid data paths

In Portkey's hosted model, requests and logs pass through Portkey's infrastructure. Its hybrid deployment runs the data plane, the gateway, in your VPC while the control plane stays in Portkey's VPC, and the gateway calls out over HTTPS to control.portkey.ai. Confirm which configuration, keys and log fields travel on that outbound path, and whether your network policy allows that egress.

Virtual keys and provider credentials

Portkey manages virtual keys and provider credentials through its control plane in hosted and hybrid setups, while Kong typically stores provider credentials in plugin configuration on your gateway. Map who can create, rotate and revoke keys under each model, and confirm whether your secrets manager can feed the gateway without sending key material to a vendor cloud.

Identity, audit and compliance evidence

Access to gateway administration, log review and audit trails depends on how identity is federated under each model. DeployOpen can prepare systems, configuration and evidence for your compliance assessment, but it does not issue audit reports or certifications. Independent auditors assess the evidence, and neither Portkey nor Kong issues a compliance attestation for your deployment on your behalf.

How an LLM request flows through Kong and the model providers

In a Kong setup, the Kong data plane sits between your applications and the model providers. Applications send their chat or embedding requests to a Kong route instead of to OpenAI, Anthropic, Bedrock or a self-hosted vLLM endpoint. The Kong Gateway process matches the route, runs the AI Proxy plugin attached to it, and forwards the request to the chosen provider with the provider credentials held in the gateway configuration, not in each application. The response travels back the same path. Because the gateway sees every request and response, it is also the natural place to decide where a request goes, and that decision is made in the gateway, not in the application.

Load balancing and fallback decisions therefore belong to the gateway configuration. With the basic AI Proxy plugin, the reader should confirm which routing behavior the Apache-2.0 edition offers for their chosen providers and request types. Multi-model balancing across round-robin, least-connections, lowest-latency and semantic strategies is documented for AI Proxy Advanced, which is part of Kong AI Gateway Enterprise. The application keeps sending the same request to the same route, so a change in routing policy is a gateway change that needs review and rollback planning, not an application release. Teams moving prompts or keys from Portkey should map each Portkey routing rule to a named Kong plugin setting before cutover, so they know which decisions move and which need an Enterprise license.

Logging and telemetry follow the same path. Kong Gateway writes its own access and error logs on the node, and any AI-specific metrics or log plugins you enable send data to the destinations you configure, such as your own log pipeline or metrics stack. Prompts and completions are only captured if you turn on logging of request bodies, so decide that explicitly with your privacy and security owners. Confirm in the plugin documentation which fields each AI plugin records and where they are sent. Because the data plane and any Konnect management connection are separate choices, check which components, if any, send telemetry outside your network before the gateway goes live.

Cost components on each side and what drives them

Cost on both sides splits into subscription or license fees, infrastructure, database and storage, observability, and the staff who run the gateway. The Kong side carries the license question: the basic AI Proxy plugin is in the Apache-2.0 gateway, while multi-model load balancing and semantic routing sit in the enterprise tier. Use the cards below to build a like-for-like quote request.

Portkey subscription or enterprise contract

Portkey's cost is a plan or enterprise contract that covers its hosted or hybrid control plane, and some gateway features sit behind that plan. This is recurring. Ask each vendor in writing which features are included at your traffic level, whether logs are priced per volume, and whether a hybrid deployment changes the fee. Confirm which of your current Portkey features are enterprise-only before comparing totals, because those are the ones a Kong build must replace.

Kong Gateway open-source infrastructure

The Apache-2.0 Kong Gateway has no license fee, so its cost is compute, load balancers and the platform team's time. Kong can run DB-less, in hybrid mode with separate control and data planes, or backed by PostgreSQL, and each topology changes how many nodes you run. Recurring costs grow with request volume and the number of high-availability zones. Ask DeployOpen for a scoped estimate of the topology you choose.

PostgreSQL and database operations

If you choose a PostgreSQL-backed Kong deployment, the database is an additional component with backups, patching, failover and capacity planning. A DB-less or hybrid data-plane design moves that work elsewhere but adds configuration distribution to operate. Ask whether your team already runs managed or self-managed PostgreSQL that can absorb this, since that decision often changes the recurring cost more than the gateway itself.

Kong AI Gateway Enterprise or Konnect subscription

Multi-model load balancing, semantic routing and the other AI Proxy Advanced algorithms need Kong AI Gateway Enterprise, which is a recurring subscription. Konnect is Kong's managed SaaS option with analytics and developer portals, and it is a separate recurring cost. Ask Kong for written pricing for only the AI features you need, and confirm whether the quote is per node, per request volume or per organization.

Observability backend and log retention

Portkey's hosted logs and tracing are an enterprise feature in its open-source gateway, so a self-hosted Kong build needs a log and metrics destination of its own. Its cost rises with prompt volume, the length of stored prompts and retention period. Decide retention before the quote, because logs containing prompts can also bring data handling and access-control work that adds effort.

Integration and migration effort

Moving applications from Portkey's routing format to Kong routes and AI Proxy is a one-time cost covering config mapping, virtual key replacement, fallback testing and cutover. Each provider you add, and each application that calls the gateway directly, adds testing work. Ask for a fixed-scope quote that lists the providers, applications and environments in scope, and whether ongoing upgrades and support are a separate retainer.

What moving LLM traffic from Portkey to Kong involves

The sequence moves gateway traffic from Portkey to Kong in controlled stages, so each application cuts over only after its routes, keys, fallbacks and logs have been checked. The end state is a Kong route serving LLM calls with logging you operate yourself.

Inventory Portkey configs, virtual keys and fallbacks
Export the list of Portkey configs, virtual keys, model targets, fallback and retry rules, and the applications that call each one. Confirm which configs use load balancing, guardrails or caching, since those behave differently in Kong's tiers. Ask the platform owner which of these settings are used in production. The step is done when every application maps to a documented config and every fallback chain is written out in plain terms.
Map configs to Kong services, routes and AI Proxy
Translate each Portkey config into a Kong service and route, then attach the AI Proxy plugin with the provider, model and credential settings it needs. Check the provider list and request types in Kong's AI Proxy documentation against the models you use, and confirm the Kong Gateway version required by each feature. The step is done when a test request through the new route returns the same response shape the application expects.
Decide which Advanced features to license
Compare the Portkey features your inventory depends on with the Apache-2.0 AI Proxy plugin. Multi-model load balancing and semantic routing sit in AI Proxy Advanced, which Kong documents as part of its AI Gateway Enterprise offering. Where a feature is not required, the open-source plugin may cover it. Request a written quote for any Enterprise component against this list. The step is done when each Portkey feature is marked as covered by open source, covered by Enterprise, or dropped by decision.
Rebuild key management and access control
Recreate Portkey virtual keys as consumer credentials and access rules in Kong, and move provider API keys into the gateway's secret store, not into application code. Confirm with Kong which budget, quota and per-consumer controls your tier provides before assuming parity. Test that a revoked key stops working and that a new key reaches the same routes. The step is done when each application uses a gateway-issued credential and no provider key remains in client configuration.
Set up logging and observability to replace Portkey logs
Portkey's hosted logs and tracing will no longer be the record of LLM calls, so choose where Kong's logs and metrics go before cutover. Route them to your existing observability stack or to a self-hosted LLM tracing tool, and confirm which fields, including token counts and model names, are captured. Agree a retention period with the data owner. The step is done when a sample request produces a trace or log entry in the destination you chose, with the fields your cost and audit reviews need.
Run in parallel and cut over applications
Send a small share of one application's traffic through Kong while Portkey continues to serve the rest, then compare responses, latency, error rates and fallback behavior between the two paths. Move applications one at a time, keeping the Portkey config available for rollback until each one has run cleanly. The step is done when every application is on Kong and the comparison records show no unexplained differences in output or failure handling.
Decommission Portkey and close out evidence
Once all traffic has moved, revoke Portkey virtual keys and provider credentials it held, remove Portkey routing from application configuration, and export any logs you still need for records through Portkey's export process. Update runbooks and access reviews so they reference Kong. Record the cutover decisions and their owners for your compliance file. The step is done when no application calls Portkey, its credentials are revoked, and the decision record is stored with the platform documentation.

Questions buyers ask before choosing Kong or Portkey for LLM traffic

Is the basic AI Proxy plugin enough without a Kong Enterprise license?

For many single-model setups, the Apache-2.0 AI Proxy plugin covers the core request path: it handles chat completions, embeddings and other request types across providers including OpenAI, Azure OpenAI, Amazon Bedrock, Anthropic, Vertex AI, Ollama and vLLM. The gap appears when you need traffic spread across several models, or routing by latency, usage or prompt similarity. Those behaviors sit in AI Proxy Advanced, which is documented as part of Kong's AI Gateway Enterprise offering. Map each Portkey feature your team uses to one of these two tiers before you decide. Confirm the current plugin requirements, such as minimum Kong Gateway versions, in the Kong AI Proxy plugin documentation.

What does the Kong Enterprise license change in cost terms?

The license is the main recurring software cost on the Kong side. It is quoted by Kong, so ask for pricing that covers AI Proxy Advanced, support tier, and any Konnect subscription if you want managed analytics or developer portals. Portkey's cost is also driven by its plan and log volume, so compare both against the same traffic forecast. Cost drivers on the self-hosted side include compute for the gateway, the database if you run one, logging storage, and the staff time to operate it. Ask both vendors for a quote built from your request volume, number of models, and retention period.

How long does a switch from Portkey to Kong take?

The duration depends on your scope, not on a standard number. The main drivers are how many applications call Portkey, how many virtual keys, budgets and fallback configs must be mapped to Kong routes and AI Proxy settings, whether you already run Kong for APIs, and how much logging history must be kept or exported. Teams that run gateways in several environments usually take longer because each environment needs its own testing. A written scope from DeployOpen, which is fixed-scope for most engagements, sets the timeline after an inventory of your traffic.

What does the team need to provide before the work starts?

Three owners are needed. First, API owners for each application or service that calls Portkey, who confirm model names, request types and expected behavior during cutover. Second, a provider key owner who can issue, rotate and store the OpenAI, Anthropic, Azure or other provider credentials that Kong will hold. Third, an observability owner who decides where traces, metrics and logs go and who receives alerts. You also need an agreed Kong environment, either an existing cluster or one to be provisioned in your own infrastructure, plus a named approver for production changes.

Can our own platform team do this migration without outside help?

A capable platform team can do it, provided it already runs Kong and has time to test each model route and failover path. The parts that usually need outside effort are the mapping of Portkey configs to Kong, load-balancing tests, secrets handling, and documentation for auditors. Ask whether you need Enterprise features before starting, because the design changes if AI Proxy Advanced is required. DeployOpen plans, implements, hardens, documents and supports self-hosted platforms in your own infrastructure, and ongoing support is available as a retainer.

How do we close logging and tracing gaps after leaving Portkey's hosted logs?

Portkey's hosted logs and tracing do not carry over automatically, so you must choose a destination before cutover. Kong emits logs through its plugin ecosystem, and you can route LLM request metadata to your existing observability stack, such as OpenTelemetry-based tooling. Decide which fields you need, including model, token counts, latency, consumer and error codes, and confirm each one can be captured in Kong. Prompt and response bodies raise privacy and retention questions, so set those rules before enabling them. Export historical Portkey logs first if you need them for audit or cost comparison.

Does data stay inside our network with Kong and with Portkey?

With a self-hosted Kong gateway, requests and responses pass through infrastructure you control, and the provider calls go directly from your network to the model provider you configured. Kong Konnect, the managed SaaS control plane, sends configuration and analytics to Kong's cloud, so avoid it if that is not acceptable. Portkey's hybrid deployment runs the gateway in your VPC, but the control plane stays in Portkey's VPC and the gateway calls out over HTTPS to control.portkey.ai. Confirm exactly which telemetry and configuration leave your network under each setup with the vendor and your security team.

When staying on Portkey still makes sense

Staying on Portkey is a reasonable choice when the team has no Kong estate to build on. Running Kong AI Gateway means operating Kong Gateway itself, with its database or DB-less configuration, control and data plane design and plugin lifecycle, and that work only pays off if the organization already carries it for its APIs. If LLM traffic is the only reason to add a gateway, a purpose-built LLM product may cost less in engineering attention, and Portkey's managed experience is designed for exactly that case.

The stronger case for Portkey is packaged functionality. Prompt management, semantic caching, advanced logging and tracing, access control and PII redaction are listed in Portkey's repository as enterprise features outside the open-source gateway, so teams that want them ready to use, rather than built from plugins and external stores, will find Portkey's product more complete. Kong's AI Proxy plugin covers request types and providers well, but teams would need to assemble prompt versioning, caching and redaction themselves. Ask Portkey to confirm which of these features are included in your plan, and whether the hybrid deployment's control plane meets your data handling rules.

Keep in mind that feature lists and plan boundaries change. Before deciding, request current written confirmation from Portkey of what each plan includes, what log retention applies, and which features run in your VPC under the hybrid model. Compare that against the Kong AI Proxy Advanced entitlements you would need from Kong's Enterprise offering, so the decision rests on the same scope in both quotes.

How DeployOpen helps evaluate and migrate an LLM gateway

DeployOpen plans, implements, integrates, hardens, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For a Portkey vs Kong AI Gateway decision, we start by agreeing in writing which gateway features your teams actually depend on, then build and test only the tier that covers them.

How we scope

Inventory which Portkey features your applications call, such as fallbacks, virtual keys, logging and prompt management.
Confirm whether your Kong estate already runs Kong Gateway and which Kong tier your AI routing needs.
Agree deliverables, timelines and exclusions in writing before any work starts, as engagements are fixed-scope.

How we build

Deploy the chosen Kong Gateway configuration with AI Proxy routes inside your own infrastructure.
Map Portkey configs and fallbacks to Kong routes, and test each provider path against your traffic patterns.
Test gateway controls, including access, rate limits and logging destinations, and record gaps against Enterprise features.

How we hand over

Document the architecture, configuration, provider credentials handling and the feature gaps versus Portkey.
Provide runbooks for upgrades, backups and cutover steps so your team can operate the gateway independently.
Prepare evidence of configuration and test results that your internal reviewers and independent auditors can examine.

How we support

Ongoing support, upgrades and continuous testing are available as a retainer after handover.
Retainers can cover Kong version upgrades, including the release line you adopt, and re-testing of routing changes.
Ask for a written quote tied to the agreed scope so every support and migration cost is visible before you commit.

Portkey vs Kong AI Gateway