Open-source alternative to Portkey prompt management

Open-source alternative to Portkey prompt management: the short answer

If your teams deployed prompts through Portkey and now need prompt management outside an enterprise contract or inside your own network, the main open-source replacement is Langfuse prompt management, used alongside a self-hosted LiteLLM gateway. Portkey's open-source AI Gateway does not include prompt management, which Portkey lists as an enterprise feature, so the gateway alone will not replace what your teams used. Langfuse stores and versions prompts centrally, uses labels to manage deployments across environments, caches prompts client-side in the SDK so retrieval adds no latency, and links prompts to traces so you can compare performance by prompt version.

The licensing split matters before you commit. Langfuse's repository is MIT licensed except for the ee/, web/src/ee/ and worker/src/ee/ directories, which fall under a separate enterprise license. Langfuse states that all core features and APIs are available in the MIT version, but protected prompt labels need an Enterprise license key on a self-hosted deployment, along with project-level RBAC roles and audit logs. Confirm which of these controls your change process relies on, and whether a license key is required before you rely on them in production.

The replacement does not do the migration or the governance by itself. You will need to export prompt text and variables from Portkey, recreate them as Langfuse prompts with variable placeholders, and update applications to fetch by label. Langfuse also depends on PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible storage, so plan who operates that stack and who approves label promotions. Ask the vendor and your own environment to confirm the export options and the current feature list for your Langfuse version before you set a cutover date.

Which Portkey prompt management capabilities map to which open-source tools

This table maps each prompt management capability in Portkey's enterprise tier to its open-source counterpart in Langfuse, LiteLLM or a Langfuse Enterprise feature. Read the rows for gaps first, because several controls need a Langfuse license key or LiteLLM Enterprise and the gateway routing layer stays with LiteLLM.

Capability	Open-source tool and feature	Coverage and gaps
Central prompt storage	Langfuse prompt management in the open-source version	Covered. Prompts are stored centrally and available through the SDK and API in the MIT-licensed version.
Versioning	Langfuse prompt versions	Covered. Each change creates a new version that can be compared and rolled back in the open-source version.
Environment labels	Langfuse labels such as production and staging	Covered for deployment across environments. Protected labels that block unauthorized edits need a Langfuse Enterprise license key.
Prompt templates with variables	Langfuse prompt templates, called from application code through the SDK	Covered for template substitution. Template syntax must be rewritten if it differs from Portkey's format, so test rendered output before cutover.
Client-side caching	Langfuse SDK prompt cache	Covered. The SDK caches prompts client-side, so retrieval adds no latency per call according to Langfuse.
Trace linkage and analysis by version	Langfuse traces linked to prompt versions	Covered if the application sends traces to Langfuse. Historical Portkey logs are not linked and need a separate export.
Access control	Langfuse project-level RBAC roles, or LiteLLM virtual keys for gateway access	Project RBAC needs a Langfuse Enterprise license key. LiteLLM virtual keys control model and budget access but do not govern who edits prompts.
Audit logging	Langfuse audit logs and LiteLLM audit logs	Both need enterprise licenses. Confirm which events each product records before relying on them for an assessment.
Gateway routing, fallbacks and keys	LiteLLM AI Gateway with routing and virtual keys	Routing stays with LiteLLM, not Langfuse. Langfuse manages prompt text only, so the gateway must be deployed separately.

Which open-source prompt management option fits which team

Prompt management can sit in a self-hosted observability platform, in a version-controlled repository, or in a gateway. Each option trades operating effort against built-in governance, so the right choice depends on how many teams edit prompts and how strict your access controls must be.

Langfuse self-hosted on the MIT core

Langfuse's core features and APIs, including prompt versions, labels and client-side caching, are available in the MIT-licensed code without feature limits. This fits teams that need versioned prompts linked to traces and can run a small set of editors under shared administration. The trade-off is operating a second data stack: Langfuse depends on PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible storage. Confirm which controls you need before you deploy, because some governance features sit behind the enterprise key.

Langfuse with an Enterprise license key

Protected prompt labels, project-level RBAC roles, audit logs and server-side data masking need an Enterprise license key on a self-hosted Langfuse instance. This fits regulated teams where production prompts must be locked to approved reviewers and changes must be traceable to a person. The trade-off is a custom-priced contract on top of your infrastructure cost, plus the same operating work as the MIT core. Ask the vendor for the written list of features covered by the key and the support terms that come with it.

Opik for a fully Apache-2.0 stack

Opik, built by Comet, is Apache-2.0 licensed, and its README states that the full platform, including prompt management, can be self-hosted without a commercial license. This fits teams that want prompt management, tracing and evaluations in one open-source project with a permissive license. The trade-off is that you must verify the governance features you need, such as role separation and audit trails, against the current documentation and your own tests, since these are not confirmed here. Evaluate it alongside Langfuse before choosing.

Git-managed prompts with LiteLLM as the gateway

Storing prompt templates as files in Git gives you pull request review, signed history and rollback without a prompt database. Applications load the approved version at deploy time, and LiteLLM handles routing, keys and budgets at the gateway. This fits teams with strong engineering review practices and few non-developer prompt editors. The trade-off is that you build the tooling for labels, A/B comparisons and per-version trace analysis yourself, and you lose the link between prompt versions and live traces unless you add it through logging to Langfuse or OpenTelemetry.

How Langfuse prompts and LiteLLM fit together at request time

Your application no longer hard-codes prompt text in the codebase. It asks the Langfuse SDK for a prompt by name and label, such as a production or staging label, and the SDK returns a stored version. The SDK caches that version client-side, so retrieval adds no latency on each call. Your code fills the template variables, then sends a standard OpenAI-format request to the LiteLLM endpoint instead of calling a model provider directly. The request carries a LiteLLM virtual key issued for your application, not a provider key.

Fetch by label: request the prompt by name and label, not by a fixed version number, so promotions in Langfuse reach the application without a redeploy.
Fill variables: compile the template with your variables in code before the call, and keep variable names stable across versions.
Send through LiteLLM: point the OpenAI-compatible base URL at the LiteLLM proxy and pass the virtual key.
Link the trace: attach the prompt version to the generation so the trace in Langfuse shows which version produced the output.

Before you cut over, confirm with your platform team which labels exist, how the SDK cache is refreshed, and what your application does if the Langfuse service is unreachable.

How to move Portkey prompt templates and variables into Langfuse

This sequence moves each Portkey prompt into a versioned Langfuse prompt with environment labels, so applications fetch prompts by label and every response can be traced back to the prompt version that produced it.

Confirm what Portkey will export
Before any engineering starts, the platform owner asks Portkey which prompt fields can be exported on your current plan, including template text, variable names, model settings and version history. Prompt management is an enterprise feature outside the open-source gateway, so the answer depends on your contract. Record the export format and any gaps in a short inventory listing each prompt ID, its owner and the applications that call it. The step is done when every prompt in production has an inventory row and a confirmed export path.
Export prompt text and variable names
The engineer who owns the prompt library exports each template's text and its declared variables from Portkey into a version-controlled folder, one file per prompt. Keep the original Portkey prompt ID and version number in the file header so each Langfuse version can be matched back to its source. Check the variable list against the placeholders that actually appear in the template text, because undeclared or misspelled variables are a common source of silent errors after migration. The step is done when every inventory row has a matching file and a variable count that agrees with the template text.
Normalize variable syntax to Langfuse format
Portkey and Langfuse may use different placeholder syntax, so the engineer rewrites each template into the format Langfuse expects and keeps the variable names unchanged wherever possible. Where a Portkey template uses logic that does not map cleanly, such as conditional blocks, the engineer splits it into a template plus application code and records that decision in the inventory. A reviewer from the application team reads each rewritten template next to the original. The step is done when a script renders every template with sample values and no placeholder is left unresolved.
Create prompts with production and staging labels
The engineer creates each prompt in self-hosted Langfuse through the SDK or API, with a commit message that names the Portkey source version. Each prompt receives a production label on the version that is live in Portkey today and a staging label on the normalized version under test. Langfuse's version control and labels let teams move a label between versions without a code change in the application. The step is done when every prompt lists both labels in the Langfuse UI and the production label renders the same text as the live Portkey template.
Point the SDK at labels instead of Portkey prompt IDs
The application team changes each call site so it fetches the prompt by name and label from Langfuse rather than by Portkey prompt ID. The Langfuse SDK caches prompts client-side, so retrieval adds no latency once the cache is warm. Keep the Portkey lookup behind a feature flag or configuration switch during the change so the old path can be restored quickly. The step is done when a staging deployment fetches every prompt by label, and the trace for a test request shows the prompt name and version.
Validate outputs against recorded responses
The QA or evaluation owner replays a fixed set of representative inputs through both the Portkey prompt and the Langfuse prompt, using the same model and settings. They compare the outputs against responses recorded from production, looking for changed structure, missing fields, tone shifts and variable substitution errors. Differences are either fixed in the template or accepted and documented as an intentional change. The step is done when every prompt has a signed-off comparison and each accepted difference has a written reason.
Cut over environment by environment
The team moves one environment at a time, starting with a non-production environment, then staging, then production. Before each move, the owner promotes the validated version to the production label and confirms that traces show the new prompt version. Keep the Portkey path available until the last environment is stable and no application still reads Portkey prompt IDs. The step is done when a search of the code and configuration finds no remaining Portkey prompt references, and an agreed rollback procedure is written down for each environment.

What changes for teams that deployed prompts through the gateway

Prompt references in code will change from a gateway-side identifier to a Langfuse prompt name and label, and that is the first place migration work shows up. Applications that asked the gateway for a prompt by its Portkey identifier need to fetch it through the Langfuse SDK instead, usually by name plus a label such as production or a version number. Inventory every call site before cutting over, including batch jobs, evaluation scripts and services owned by other teams, because a missed reference will keep pointing at a prompt that no longer receives updates. Plan the change as a code release, not a configuration edit, and confirm with your Langfuse SDK version which fetch options and client-side caching settings apply in your language.

Rollbacks change shape. In a label-based model, reverting a bad prompt means moving the production label back to the previous version, which can be done without redeploying any application, provided the SDK is not pinned to a specific version number. Teams should decide who may move labels, and whether protected prompt labels are needed to block unreviewed moves. Protected prompt labels are an Enterprise license key feature in self-hosted Langfuse, so confirm whether your approval process depends on them before you design around them. Practice one rollback in staging so the on-call team knows which label to move and how long the SDK cache takes to pick up the change.

Audit trails and access control are not a like-for-like copy of what you had. Portkey's prompt management sits in an enterprise tier, so the audit model you relied on there should be mapped control by control, not assumed. In self-hosted Langfuse, project-level RBAC roles and audit logs are Enterprise license key features, so confirm which roles your editors and reviewers need before go-live. Gateway-side rendering is also lost if you used it: the template and variable substitution now happen in application code or the SDK, so teams must agree on one rendering path and test that variables resolve identically. Finally, because SDK versions in each service set how prompts are fetched, coordinate SDK upgrades across all services that consume prompts, and track the version of each one in your change log.

Running Langfuse prompts beside the rest of your LLM stack

Once Langfuse holds your prompts and LiteLLM handles routing, budgets and keys, the two systems share work that can drift apart. The cards below cover the four decisions that determine whether that split stays manageable: where configuration lives, who controls labels, how traces are joined, and which controls need an enterprise license.

Duplicate configuration

Model names, fallback chains and provider settings can live in both the LiteLLM proxy config and the prompt definitions in Langfuse. Pick one owner for each setting and write it down, so a model change in one place does not silently conflict with a prompt that assumes another model. Ask your team which file or prompt field is authoritative for model selection, and whether a prompt pins a model at all. Review both systems together during every provider change.

Label governance

Langfuse uses labels to move prompt versions between environments, so a label such as production decides what live traffic runs. Without restrictions, any project member with prompt write access can repoint that label. Self-hosted protected prompt labels are an Enterprise license feature, so check whether your license covers them before relying on them as an approval gate. If they are not available, document who may promote a version and record each promotion in your change process.

Trace identifiers across systems

A single user request may pass through the application, the LiteLLM proxy and Langfuse, each with its own identifiers. Agree on one trace or session ID that the application sends and that LiteLLM forwards to Langfuse, so a prompt version can be tied to the gateway call that used it. Test this with a sample request and confirm the ID appears in both the gateway log and the Langfuse trace before cutover, since missing links make cost and quality analysis by prompt version unreliable.

Enterprise license scope for SSO

LiteLLM's admin UI SSO is free for a small number of users, after which an Enterprise license is needed, and Langfuse gates RBAC roles, audit logs and SCIM behind its own license key. Each product needs its own license decision, so the two can differ. Map every user and role in both systems to your identity provider, and confirm in writing with each vendor which controls are included at your user count. Do not assume that one license covers the other.

What does replacing Portkey prompt management cost and require?

What are the cost components of replacing Portkey prompt management with Langfuse?

The cost has five parts: infrastructure, operations time, any Langfuse Enterprise license, migration effort, and optional support. Langfuse's core prompt management features are in the MIT-licensed open source version, so the software itself carries no license fee. Infrastructure covers the self-hosted stack: Langfuse Web, Langfuse Worker, PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage. Operations time is the ongoing work of patching, backups and upgrades. Migration effort covers moving prompt templates and variables. Ask any vendor or partner for a quote that lists each component separately, with the cloud or data center assumptions stated in writing.

Which costs recur and which are one-time when self-hosting Langfuse prompts?

Infrastructure and operations recur every month, because the database, ClickHouse, cache and blob storage run continuously and need patching, monitoring and backups. An Enterprise license key, if you need protected prompt labels or other gated features, is usually a recurring contract. Discovery, migration of prompts, SDK changes in applications and cutover testing are one-time costs. Major upgrades, such as the v3 to v4 path, are periodic one-time efforts that repeat each time Langfuse changes major version. Ask the quote to separate one-time project fees from any monthly retainer for support and upgrades.

What drives the cost of self-hosted Langfuse infrastructure up or down?

Infrastructure cost follows trace volume, prompt count and availability requirements. Langfuse's scaling guide lists minimum resources of 2 CPU and 4 GiB each for web, worker and PostgreSQL, 1 CPU and 1.5 GiB for Redis or Valkey, and 2 CPU and 8 GiB for ClickHouse, with at least 16 GiB of ClickHouse memory for larger deployments. These are starting points, not sizing for your load. Heavy ingestion can require splitting langfuse-web into separate ingestion and user interface deployments and scaling workers on CPU load. Docker Compose on a single VM suits testing only because it has no high availability, so production sizing adds cost. Ask for the sizing assumptions in the quote.

How long does it take to move prompts from Portkey to Langfuse?

Duration depends mainly on how many prompts exist, how many environments (such as development, staging and production) they run in, and how many applications fetch them. A small set of prompts in one environment is a much shorter task than dozens of prompts with versioned history across several environments. Time also depends on how many applications change from fetching prompts through the gateway to fetching them from Langfuse, and on how much parallel testing you require before cutover. No duration is fixed by the product, so ask for a timeline written against a defined scope with an inventory of prompts, environments and applications.

What must our team supply before a prompt migration can start?

Your team needs to supply an inventory of every prompt deployed through Portkey, including template text, variables, versions and the environments that use them. You also need named owners for each application that consumes prompts, access to the cloud account or data center where Langfuse will run, and a decision on storage for blobs and database backups. Security needs to approve where the PostgreSQL, ClickHouse and blob data will sit, and someone must own the Langfuse license key decision if Enterprise features are required. Confirm each of these with your own environment before you agree a written scope.

Can our own engineers replace Portkey prompt management without an outside partner?

Yes, if your team already runs PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage in production and has Kubernetes or Docker operations experience. The official documentation covers Docker Compose for testing and Helm or Terraform deployments for production, and the upgrade guides describe the migration steps. The harder parts are sizing ClickHouse for your data, planning the v3 to v4 path if you start on an older version, and keeping backups and patching current. Teams without that experience usually find the operating load larger than the software setup. An outside partner can help with a fixed-scope implementation, and DeployOpen offers that on written scope, with ongoing support available as a retainer.

What version and dependency requirements must we check before deploying Langfuse?

Check the Langfuse v3 to v4 guide before you choose versions. It requires ClickHouse 25.12 or later (26.4 recommended), PostgreSQL 15 or later and Redis 7.0 or later. The guide also advises about 3x ClickHouse disk headroom when backfilling historic data, which can be a significant storage cost if you migrate large histories. Confirm these requirements against the release you plan to deploy, because the release notes are the authoritative source and may change.

How DeployOpen scopes and delivers prompt management migrations

DeployOpen plans, implements, integrates, hardens, backs up and documents self-hosted Langfuse and LiteLLM inside your own infrastructure, whether that is your cloud accounts, data centers or an air-gapped network. Before any work starts, the scope is agreed in writing, covering which prompts move, which environments and labels are created, how the LiteLLM gateway calls Langfuse, which keys and roles are issued, and what acceptance tests end the engagement. Most engagements are fixed-scope, so deliverables and timelines are fixed in that document rather than estimated later. Ask for the same level of detail in any quote you receive, including who owns the prompt inventory and who signs off on each cutover step.

Delivery typically covers the infrastructure the platforms depend on: Langfuse Web and Worker, PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible blob storage, deployed on Docker Compose for a non-production test or on Kubernetes with Helm or Terraform for production. DeployOpen also configures the LiteLLM proxy, its PostgreSQL database and shared Redis state, and secret handling for the master key. Backups of the stores, restore tests and the documentation of runbooks are part of the handover, so your own operators can run the platform after the project closes. Confirm with DeployOpen which of these items sit inside the fixed scope and which are optional.

Once the platform is live, upgrades and ongoing support are usually handled under a retainer. Langfuse major versions follow separate migration guides, and LiteLLM publishes a support window for its recent minor versions, so a retainer should name who tracks those releases, who tests them in a non-production environment and how long a rollback is kept available. Ask whether the retainer includes security testing of the gateway and the prompt store, and how the retainer scope changes if you add providers, teams or regions. These questions set the recurring cost more accurately than any headline rate.

Open-source alternative to Portkey prompt management