LiteLLM vs Kong AI Gateway

LiteLLM's admin UI supports SSO with Google, Microsoft Entra ID (Azure AD), Okta and generic OIDC providers, and from v1.76.0 that SSO is free for up to 5 users. A small platform team that administers the gateway for a handful of people can usually stay on the MIT edition and use virtual keys and budgets to control spend per user, team or project. Before relying on the free tier, confirm that your deployed version is v1.76.0 or later and that your identity provider's sign-in flow works for every administrator who needs access. Kong takes a different route to identity. Authentication and access control are set through its plugin model and attached to services and routes, so the identity design belongs to the team that operates Kong, not to a separate admin console.

The 5-user limit is the point where a multi-team organization hits LiteLLM's licensing boundary. Beyond that count, SSO requires an Enterprise license, and the same tier adds SCIM provisioning, audit logs, organization and team RBAC, JWT authentication with identity provider tokens and IP allowlisting. SCIM matters when people join, change teams or leave often, because deprovisioning then happens in the identity provider rather than by hand in the gateway. Audit logs matter when someone must show who changed a team budget or a key's model access. Map each business team to a LiteLLM team before purchasing, and test whether your identity provider's group claims reach the gateway as you expect.

The governance question for a Kong shop is different. If your organization already uses Kong for APIs and has centralized authentication there, extending the same controls to LLM routes may matter more than the LiteLLM license boundary. Confirm with Kong which authentication and identity plugins apply to AI routes in your edition, whether SCIM-style provisioning or audit trails exist, and which of these sit in the AI Gateway Enterprise offering. Load balancing and semantic routing are also part of that enterprise scope, so check the AI Proxy Advanced plugin documentation alongside any identity requirements when you build the quote.

The LiteLLM side of the project runs long when the database and cache are treated as afterthoughts. LiteLLM's production guide names PostgreSQL as the primary supported database, and virtual keys need PostgreSQL, so the budget and key data have to live there from day one. Once more than one proxy instance runs, Redis 7.0 or newer becomes required for shared rate limits, router state and cache. Plan the LITELLM_MASTER_KEY handling early: it must start with sk-, belongs in a secret manager, and every admin workflow depends on it. Spend updates are batched to the database every 60 seconds at scale, so budget figures can lag real usage. Confirm with your own platform team how that lag is acceptable for your chargeback rules, and plan a key migration path for any teams already calling providers directly with their own credentials.

On the Kong side, the first schedule risk is version alignment. The AI Proxy plugin needs Kong Gateway 3.6 or later, AI Proxy Advanced needs 3.8 or later, and some features need 3.10 or 3.11. Check the exact feature list against the release you deploy before you write plugin policy, because a design that assumes semantic routing or load balancing on a lower line will fail in test. Policy design is the second cost: decide which consumers reach which models, where rate limits sit, and how hybrid control and data planes are split, since the topology choice affects where configuration and secrets live. Kong's enterprise tier also gates load balancing and semantic routing, so get the licensing answer in writing before design begins. Three areas overrun most often. Identity integration is first: LiteLLM's free SSO covers up to 5 users, and beyond that the Enterprise license, SCIM and JWT handling must be scoped and bought. Budget reconciliation is second, because provider invoices, gateway spend records and team chargeback numbers rarely match without a defined owner and a reconciliation schedule. The third is the enterprise feature decision made late, after architecture is built around open-source assumptions. Your team must supply the user count, the identity provider, the existing Kong footprint, the model providers in use, the audit and retention requirements, and a named owner for budgets. DeployOpen scopes that work in writing before any build starts.

DeployOpen is a Chainscore Labs company that plans, implements, hardens and supports self-hosted open-source platforms inside the client's own infrastructure, including their cloud accounts, data centers and air-gapped networks. For an LLM gateway decision, that means the work can start before any platform is chosen: we can help you test whether a shared gateway fits your model providers, identity setup and audit needs, and then carry the chosen option through deployment. DeployOpen does not favor LiteLLM or Kong AI Gateway. Our advice follows the written assessment, and if the assessment points to a different component or to no gateway at all, that is the recommendation we would put in writing.

Every engagement is fixed-scope, so the scope comes from the assessment rather than from a sales template. A useful starting point is a short list of decisions: whether you already operate Kong Gateway and which version, which providers and request types must be routed, whether per-team budgets and virtual keys are required, and whether SSO beyond the free LiteLLM tier or Kong's Enterprise-only load balancing and semantic routing is a requirement. The written scope then names the deliverables, such as a PostgreSQL and Redis design for the LiteLLM proxy or a Kong plugin configuration and upgrade path, along with timelines and acceptance criteria. DeployOpen does not publish a price list, so the quote follows from that scope. Ask any vendor, including DeployOpen, to itemize licenses, subscriptions, infrastructure and services separately.

Ongoing work is available through retainers covering support, upgrades and continuous testing, which matters for a gateway that sits in front of every model call. Ask for the scope of each retainer in writing, including how the team tracks upstream releases, how it tests a gateway upgrade before rollout, and who owns the keys and budgets after handover. Confirm with the vendor which features your chosen edition includes, because license terms can change between releases. DeployOpen prepares systems, controls and documentation for your compliance assessments, but it does not issue audit reports or certifications. Those come from independent auditors and certification bodies, so plan that evidence collection with them early.