How to use this LLM observability requirements checklist
Before an LLM feature goes live, you need a way to record prompts, outputs, evaluations and costs in storage you control. This section explains how to use the checklist that follows. Treat it as a set of requirements your security, compliance, data and application teams agree in writing before anyone selects Langfuse, Phoenix or another tool. Each requirement needs a named owner and a verification step, such as a test trace sent from a staging application, a retention setting checked in the running system, or a restore of the trace store from backup. A requirement that cannot be verified is a gap, not a decision.
Licensing comes first because it decides which controls you can actually have. Langfuse's core is MIT licensed, but its ee directories fall under a separate enterprise license, and several controls such as project-level RBAC roles, data retention policies, audit logs and server-side data masking need an Enterprise license key on a self-hosted deployment. Arize Phoenix is licensed under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service. Confirm with each vendor, in writing, which features your deployment model and user count require, and map every control in the checklist to a license tier before you compare products. Ask for the license terms that apply to your own use, not the marketing summary.
Work through the checklist in the order it appears, and record the answer to each item alongside its evidence. Where a requirement depends on a license tier, mark it as blocked until the license decision is made rather than assuming a workaround exists. The outcome should be a signed requirements baseline that a partner can quote against, with each open item tied to a decision owner. This keeps the scope of an implementation fixed and makes it clear which costs are one-time setup and which recur, such as license fees, storage growth and upgrade work.
Functional requirements: what to trace, how much, and for how long
This table lists the functional requirements an LLM observability platform must meet, why each one matters, and how to verify it before go-live. Read each row as a question to answer in your own environment, then use the verification step as an acceptance test with your implementation partner.
| Requirement | Why it matters | Verification step |
|---|---|---|
| Applications and frameworks to instrument | Each chat app, RAG service, agent or batch job that calls a model needs its own trace source, or coverage gaps appear silently. | Inventory every LLM call path, then confirm each one emits a trace in a test environment before scope is signed off. |
| Trace and observation volume | Ingestion volume drives ClickHouse disk, memory and worker load, and agent workloads produce many more spans per request. | Run a load test with representative traffic, including your longest prompts and deepest agent chains, and measure storage growth per day. |
| Retention periods by data class | Retention must match the business, regulatory and privacy duties for each trace type, and AI Act Article 19(1) sets a six-month minimum for logs of high-risk systems. | Write a retention schedule per project, confirm it with legal and compliance, and test that expired data is actually removed. |
| Masking of prompts and outputs | Prompts and outputs often contain personal or confidential data, so unmasked traces become sensitive stores in their own right. | Send seeded test values (illustrative: fake card or patient identifiers) through each instrumented path and confirm they are masked before storage. |
| Evaluations and datasets | Scores and datasets show whether quality changes between releases, and they need owners who review results before a prompt or model change ships. | Confirm a named reviewer, a scoring method per use case and a dataset export that can be reproduced from stored traces. |
| Prompt management and versioning | Recording which prompt version produced an output is needed to explain behavior and to roll back safely. | Change a prompt in a test project and confirm the trace records the new version identifier. |
| Cost tracking per project and user | Model spend and attribution to teams or tenants are needed for budgets, chargeback and anomaly detection. | Make calls under two test projects and confirm cost and token totals split correctly in the reporting view and exported data. |
| Access by project role | Trace data often holds client or employee content, so viewing rights must follow the same boundaries as the source system. | Sign in with a test account for each role and confirm it cannot see traces from projects it does not own. |
Security requirements for trace data that contains prompts and outputs
Trace data often holds the full prompt, retrieved documents and model output, so it needs the same access and protection rules as the source data. The controls below decide who can read traces, what gets masked, and how the platform is isolated and configured. Confirm each one in your own environment before you assume it is present.
Single sign-on and identity mapping
Decide early whether engineers, reviewers and auditors sign in through your corporate identity provider or through local accounts. Langfuse documents SSO options in its authentication guide, but the mapping from groups to projects and roles is your design work. Confirm which SSO protocols the version you deploy supports, how group membership is refreshed, and what happens to access when someone leaves. Local break-glass accounts should be documented and tested.
Project-level roles and enterprise scope
Langfuse documents project-level RBAC as a feature that needs an Enterprise license key, so the role model is a licensing decision as well as a security one. Without it, the open source edition should be tested for what it can separate, such as organizations and projects, before you rely on it for client or patient data. Write down which teams may view raw traces, which may only see scores, and who administers the instance.
Audit logs and retention policies
Audit logs that record who viewed or exported traces, and retention policies that delete them on a schedule, are both listed by Langfuse as Enterprise license features. Set your retention period from legal and regulatory duties, not from storage cost. For EU AI Act record-keeping, Article 19 sets a minimum of six months unless other law requires longer. Ask how deletion reaches ClickHouse and blob storage copies, not only the UI view.
Server-side masking and what it covers
Server-side data masking is another Enterprise license feature in Langfuse. Client-side masking in the SDK or OpenTelemetry Collector works without it, but it depends on every instrumented application being updated. Test with synthetic personal data and check that masked values do not reappear in exports, evaluation datasets or cached prompt versions. Mask before data reaches storage wherever possible, because deleting a stored value later is harder.
Encryption in transit and at rest
Traffic between browsers, the Langfuse Web and Worker containers, PostgreSQL, Redis or Valkey, ClickHouse and S3-compatible storage should use TLS where the components support it. Encryption at rest depends on the storage layer you choose, such as disk encryption on the ClickHouse nodes and server-side encryption on the bucket. Confirm the key management owner, the rotation process and whether backups are encrypted with the same keys.
Network isolation and secrets handling
Keep the observability stack on a private network segment, with only the ingestion endpoint and the user interface reachable from the users or applications that need them. Database credentials, the Langfuse encryption key, SSO client secrets and object storage keys should come from a secrets manager, not from Compose files or Helm values committed to source control. The OpenTelemetry Collector's authenticator extensions can check credentials on incoming telemetry, so use them where collectors sit outside the trusted zone.
Verifying controls in your environment
Documentation describes what a feature can do, not how your deployment behaves. Ask the implementation partner for test evidence for each control: a user without project access cannot open a trace, a deleted trace is removed from the storage tiers, and a masked field never reaches ClickHouse. Record the results in your control register so an auditor can trace each requirement to a test. Repeat the checks after every major upgrade.
What ClickHouse and blob storage the traces will need
A self-hosted Langfuse deployment is a set of components, not a single container. Langfuse Web serves the UI and APIs, Langfuse Worker processes events asynchronously, and the platform depends on PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage. ClickHouse holds the high-volume trace and observation data, PostgreSQL holds transactional metadata such as projects and users, and blob storage keeps raw event payloads. Each one needs its own owner, backup plan and capacity estimate before the platform team agrees to run it in production.
The Langfuse scaling guide lists minimum resources of 2 CPU and 4 GiB for web, worker and PostgreSQL, 1 CPU and 1.5 GiB for Redis or Valkey, and 2 CPU and 8 GiB for ClickHouse, with at least 16 GiB of ClickHouse memory for larger deployments. Treat these as floors. Langfuse recommends scaling worker containers on CPU load and, under heavy ingestion, splitting Langfuse Web into separate ingestion and user interface deployments. Ask the partner which of these splits they expect for your ingestion profile and whether the chosen Kubernetes or Helm layout supports them from day one.
If you plan to move to Langfuse v4, the upgrade guide requires ClickHouse 25.12 or later (26.4 recommended), PostgreSQL 15 or later and Redis 7.0 or later, and moves to an observations-first ClickHouse data model. It also advises about 3x ClickHouse disk headroom when backfilling historic data. Request a sizing proposal that states expected trace and observation volume per day, the retention period, the backup and restore approach for ClickHouse and PostgreSQL, and how blob storage lifecycle rules match your retention decision. Ask for a tested restore time, not only a backup schedule.
Integration requirements: OpenTelemetry, frameworks and identity
Traces only help if they arrive from every application, pass through a controlled path, and are visible only to the right people. The cards below cover the instrumentation layer, the Collector that sits between applications and storage, and the identity controls that govern who can see what.
Instrument through OpenTelemetry
Decide whether each application sends traces through the Langfuse or Phoenix SDK, through OpenTelemetry directly, or both. OpenTelemetry is vendor-agnostic and does not store data itself, so it keeps the instrumentation portable if the backend changes. Ask each partner to list which frameworks and languages are covered natively and which need custom spans. Confirm that every application team uses the same attribute names for user, session and model so cost and quality reports can be joined across services.
Route traces through the Collector
The OpenTelemetry Collector is Apache-2.0 licensed and receives, processes and exports telemetry from many sources. Run it as an agent beside workloads for local buffering, as a gateway for central control, or both. The gateway is the natural place to enforce authentication on incoming connections, batch and retry exports, and apply redaction before data reaches ClickHouse. Ask who owns the Collector configuration and how changes are reviewed.
Authenticate receivers with OIDC
Collector authenticator extensions can check incoming connections on receivers, for example with OIDC, and attach credentials to outgoing exporter requests, for example with the oauth2client extension. Without them, any workload that can reach the endpoint can write traces. Confirm which applications use workload identity, which use shared tokens, and how tokens are rotated. Test that a request without a valid token is rejected before go-live.
Mask data in the Collector
Client-side masking in the SDK or the Collector removes personal data before it is written to storage, which matters because prompts and outputs often contain names, account numbers or health details. Server-side masking in Langfuse is listed among features that need an enterprise license key, so buyers should confirm which masking runs where and who maintains the rules. Write a sample prompt containing test identifiers and confirm the stored trace no longer contains them.
Connect single sign-on and project roles
Staff who review traces should sign in with the organization's identity provider rather than local accounts. Langfuse documents authentication and SSO options, but project-level role-based access control and SCIM provisioning need an enterprise license key. Map who may see which projects, decide whether reviewers can open raw prompts, and set an offboarding procedure that removes access when staff leave. Confirm the identity provider protocols each product supports before committing.
Check component stability before standardizing
Trace APIs and SDKs are stable in most languages, including Go, Java, JavaScript and Python, but log SDK maturity varies and several languages are still in development. Core Collector components also have mixed stability levels, documented in each component's README. Before standardizing, list every language and Collector component in use, read its stability level, and record which ones carry a pinned version. Ask the partner how beta components will be replaced if they change.
Operational requirements and how to verify them
This table lists the operating requirements for a self-hosted Langfuse deployment and how to check each one before go-live and during operation. Read each row as a question to put to your team or partner, with the verification step that produces evidence. The main takeaway is that a working deployment is not enough: upgrades, restores, capacity and support must be tested on a schedule.
| Requirement | What to verify | Evidence to keep |
|---|---|---|
| Upgrade cadence | Updates within a major version apply database migrations automatically at startup, while major upgrades (v1 to v2, v2 to v3, v3 to v4) follow dedicated migration guides. Confirm which path each planned release takes. | Written upgrade plan naming the target version, the migration guide used and the release notes reviewed |
| Staged major upgrade | Rehearse each major migration in a non-production copy with production-like data volume. The v3 to v4 guide requires ClickHouse 25.12 or later (26.4 recommended), PostgreSQL 15 or later and Redis 7.0 or later, and advises about 3x ClickHouse disk headroom when backfilling historic data. | Staging upgrade log, measured disk headroom before and after, and a signed rollback decision |
| Backup and restore testing | Back up PostgreSQL, ClickHouse and S3-compatible blob storage together and restore them into a separate environment. Time the restore and confirm that traces, prompts and evaluation scores are queryable afterward. | Restore drill report with the backup timestamp, restore duration, row count checks and a list of gaps |
| Capacity review | Compare actual CPU, memory and ingestion against the minimums in the scaling guide (2 CPU and 4 GiB for web, worker and PostgreSQL; 1 CPU and 1.5 GiB for Redis or Valkey; 2 CPU and 8 GiB for ClickHouse, with at least 16 GiB of ClickHouse memory for larger deployments). Scale workers on CPU load and split langfuse-web under heavy ingestion. | Monthly capacity report with ingestion volume, storage growth and scaling decisions |
| Monitoring of the observability stack | Monitor the health of Langfuse Web, Langfuse Worker, PostgreSQL, ClickHouse, Redis or Valkey and blob storage, plus the OpenTelemetry Collector if one is deployed. Alert on ingestion lag and failed exports, since missing traces are an audit gap. | Alert rules, dashboards and a test showing an alert fires when a dependency is stopped |
| Single-VM versus production topology | Docker Compose on a single VM is documented for testing with no high availability. Confirm that production runs on the Kubernetes deployment path with the redundancy your recovery targets require. | Architecture decision record and a topology diagram approved by the platform owner |
| Incident response | Define who is paged for ingestion stops, failed upgrades and data loss, and how a trace gap is reported to compliance owners. Rehearse at least one scenario each year. | Runbooks, contact list and a tabletop exercise record |
| Support tier | Check whether the Langfuse Self-Hosted Enterprise plan, which adds a named support engineer, solutions architect access and a support SLA, matches the response times your business requires. Get the SLA terms in writing from Langfuse, since DeployOpen does not set them. | Signed support order form with the SLA document attached |
| Ongoing support and implementation partner | Confirm which upgrades, restore drills, capacity reviews and monitoring tasks sit in the implementation scope and which sit in a retainer. Ask for the named owner of each. | Responsibility matrix in the written scope and the retainer terms |
Questions that separate LLM observability platforms
Which Langfuse enterprise features do we need, and are they licensed for our deployment?
Start by listing the controls your policies require, then check each one against the license boundary. Langfuse states that core features and APIs are available in the MIT-licensed open source version, but several features need an Enterprise license key on a self-hosted deployment, including project-level RBAC roles, protected prompt labels, data retention policies, audit logs, server-side data masking, the org management API with SCIM, and the instance management API. Ask the partner to map each requirement to the open source tier or the license key, confirm who holds the license, and state what changes if the license lapses. Also confirm that the repository's ee directories fall under a separate enterprise license, so the code you deploy matches the terms you accepted. The Langfuse license key documentation is the natural starting point for this check.
Does Phoenix's Elastic License 2.0 limit on hosted services affect our plan?
It can, depending on who operates the service and who uses it. Arize Phoenix is licensed under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service. Running Phoenix inside your own organization for your own teams is a different case from offering LLM tracing to customers or to a partner's clients, which would require legal review of the license terms. Ask the partner to put the intended users, tenants and operators in writing, have counsel confirm the reading of the license, and document the decision alongside your architecture. If your product serves external customers, the license question may decide between Phoenix and Langfuse before any technical comparison begins.
How is sensitive data masked before traces reach storage?
Masking should happen before data leaves the application or collector, not after it lands in ClickHouse. The main options are client-side masking in the SDK, masking in the OpenTelemetry Collector before export, and server-side masking in Langfuse. Server-side data masking is listed among the self-hosted features that need an Enterprise license key, so ask whether your plan depends on it. Ask the partner to show which fields are masked (prompts, completions, tool arguments, metadata), how pattern rules for personal identifiers are tested, and what happens to content that was stored before a rule was added. Request a test that sends known sensitive values through the pipeline and shows what appears in the stored trace.
How are evaluations and datasets versioned so scores stay comparable?
Ask for a written rule that ties each score to the exact dataset version, evaluator definition and prompt version that produced it. Without that link, a score change may reflect a changed test set rather than a changed model or prompt. Confirm where datasets are stored, how they are exported and whether dataset changes are logged with an author and date. Ask who may approve a new evaluator or edit a reference answer, and how model-based scorers are pinned to a specific model version. Also ask how evaluations run in air-gapped or cost-limited environments, because the scoring model itself becomes a dependency that needs its own version control and storage.
How is ClickHouse sized, upgraded and backed up for our trace volume?
Sizing depends on trace volume, span count per request, retention period and how many users query the data at once. Langfuse's scaling guide lists minimum resources of 2 CPU and 8 GiB for ClickHouse, with at least 16 GiB of memory for larger deployments, and its v3 to v4 upgrade guide requires ClickHouse 25.12 or later and advises about 3x disk headroom when backfilling historic data. Ask the partner for a sizing estimate based on your measured traffic, a backup method for ClickHouse and object storage that is tested with a restore, and the procedure for major upgrades. Ask also who owns the disk capacity alerts. Trace data grows quickly with agent workloads, so storage planning should be part of the initial scope, not a later change.
Does OpenTelemetry ingestion work with the frameworks our teams already use?
Ask the partner to list each application, framework and language in scope and to show the instrumentation method for each one. OpenTelemetry is a vendor-neutral framework for generating and exporting traces, metrics and logs, and its trace APIs and SDKs are stable in most languages, including Go, Java, JavaScript, Python and .NET. Log SDK maturity varies, and core Collector components have mixed stability levels documented in each component's README. Ask whether LLM-specific attributes are captured, whether the Collector authenticates incoming connections with an extension such as OIDC, and how a sample request from each application appears in the stored trace. Frameworks without a native instrumentation package usually need manual spans, which adds implementation effort that should appear in the written scope.
What does the partner cover, and what can our own team run after handover?
Most engagements are fixed-scope, so the written scope should list the applications to instrument, the Langfuse features to enable, the ClickHouse and storage sizing work, the backup and restore test, the upgrade procedure and the handover of runbooks. Costs come from the partner's implementation effort, the infrastructure you run (PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage), any enterprise license, and the support you choose. Ask which parts recur, such as upgrades and support retainers, and which are one-time. Ask the partner to name the skills your team needs to operate the stack, and to state which tasks remain your responsibility after go-live.
Which compliance evidence self-hosted traces support
Traces are the raw material for AI Act record-keeping. Article 12(1) of the EU AI Act requires high-risk AI systems to technically allow automatic recording of events (logs) over the system's lifetime, and Article 19(1) requires providers to keep those logs for at least six months unless other law sets a longer period. A self-hosted Langfuse deployment can capture the request, model, prompt version, output, latency, cost and evaluation scores that such logs would contain, with the records written to PostgreSQL, ClickHouse and S3-compatible storage that the organization controls. Whether a given system is high-risk, and which events must be logged, is a classification decision for the organization and its legal advisers, not something the tracing tool settles. Read the AI Act Article 12 text and the AI Act Article 19 text with counsel before fixing the event schema.
The organization still has to define the rules the evidence depends on. Before go-live, the team should write down retention periods for each trace class, who may view traces that contain personal or client data, how erasure requests are executed across ClickHouse, blob storage and backups, and who signs off on evaluation results before a prompt or model change ships. Project-level roles, data retention policies, audit logs and server-side masking are features that Langfuse lists as requiring an Enterprise license key, so confirm early which of these rules the open-source tier can enforce and which need a license. Check the license terms with Langfuse directly, since the key-gated feature list can change between releases.
DeployOpen prepares systems and evidence but does not certify them. DeployOpen can configure the deployment, document retention and access settings, set up backup and restore tests for the trace stores, and assemble the operating records an assessor will ask for. It does not issue audit reports, attestations or certifications; independent auditors and certification bodies do that work. For the detail on client-side masking and erasure, see the GDPR masking topic, and for how logging maps to high-risk obligations, see the EU AI Act logging topic.
How to run the requirements workshop before selecting a platform
This sequence turns a loose wish list into written requirements that a platform choice and an implementation partner can be measured against. Each step closes with an artifact that the next step depends on.
Inventory applications and frameworks
Start by listing every LLM application, agent, chat tool and gateway that will send traces, and the frameworks each one uses to call models. Engineering leads for each product, with the platform team, contribute the list. The output is an inventory that names the SDK or OpenTelemetry path for each application, the hosting location and the model providers it calls. The step is done when every application has an owner and a named instrumentation approach, even if that approach is still undecided.
Agree trace volumes and retention with security and legal
For each application in the inventory, estimate how many requests it serves and how many spans a typical request produces, then agree how long traces must be kept. Product owners, security, legal and records management contribute, and the EU AI Act retention rules apply where a system is high-risk. The artifact is a volume and retention table per application. Ask your implementation partner how that table translates into ClickHouse and object storage sizing, since storage needs follow from it.
Classify prompt and output data
Work out which prompts, retrieved documents, model outputs and tool results carry personal data, client data, health data or secrets. The data protection officer, information security and the business owner contribute. The output is a classification register that maps each application to a data class and states whether masking must happen before data leaves the application or inside the collector. The step is complete when each data class has an agreed handling rule that the platform must support.
Decide which controls need enterprise features
Compare the controls from the classification register with the self-hosted feature set. Some controls, such as project-level roles, data retention policies, audit logs, server-side masking and SCIM, are documented as requiring an enterprise license key for self-hosted Langfuse, so confirm this against the current documentation. Security architects and procurement contribute. The artifact is a decision record that says, for each control, whether it is met by open-source features, by enterprise features, by client-side masking or by a documented exception.
Define evaluation ownership
Decide who creates evaluators, who reviews scores, who maintains datasets built from production failures, and who approves a prompt or model change before it ships. Product managers, quality leads and the model risk or AI governance function contribute. The output is a responsibility matrix with named roles and the evidence each review leaves behind. The step is done when every production application has a named owner for evaluation and a documented approval path.
Confirm identity, access and project boundaries
Record how engineers, reviewers and auditors will sign in, which project or team each trace belongs to, and which people may view traces that contain sensitive data. The identity team and application owners contribute. Confirm which SSO methods the chosen platform supports in your version and which roles require a license key. The artifact is an access model that lists roles, projects and the reviewers for each class of trace data.
Map OpenTelemetry and collector requirements
Document where the OpenTelemetry Collector will sit, whether as an agent beside services or as a gateway, which receivers and exporters are needed, and how the collector authenticates incoming and outgoing connections. Platform engineers and the security team contribute. The output is a collector design note that lists the components, their stability levels from the component README files, and the masking steps that run before export.
Write acceptance tests for each requirement
Turn every requirement into a test that a candidate platform and partner must pass in a test environment, such as a trace from a named application arriving with the agreed masking applied, retention deleting data on schedule, and a reviewer lacking access being refused. Quality, security and the implementation partner contribute. The closing artifact is a signed acceptance test list that becomes part of the written scope before work starts.
Frequently asked questions about LLM observability requirements
What drives the cost of self-hosted LLM tracing?
The cost comes from four areas: the infrastructure your team runs, the instrumentation work in each application, any Enterprise license you need, and ongoing upgrades. A self-hosted Langfuse deployment runs Langfuse Web and Langfuse Worker, plus PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage. The Langfuse scaling guide lists minimum resources of 2 CPU and 4 GiB for web, worker and PostgreSQL, 1 CPU and 1.5 GiB for Redis or Valkey, and 2 CPU and 8 GiB for ClickHouse, with at least 16 GiB of ClickHouse memory for larger deployments. Infrastructure recurs monthly, while instrumentation is mostly one-time per application. Ask any partner for a quote that separates these lines and states which Enterprise features are included.
Does self-hosted Langfuse need a paid license?
The core features and APIs are available under the MIT-licensed open source version without limits, according to Langfuse. Certain features need an Enterprise license key, including project-level RBAC roles, protected prompt labels, data retention policies, audit logs, server-side data masking, the org management API with SCIM, and the instance management API. Confirm the exact list against the Langfuse license key documentation before you finalize requirements, because features can move between tiers. Langfuse's self-hosted pricing page lists a custom-priced Self-Hosted Enterprise plan, so ask for that price in writing rather than assuming it.
How long does a scoped LLM observability implementation take?
Duration depends on the number of applications to instrument, the frameworks they use, the masking rules you need, whether SSO and project roles are in scope, and whether ClickHouse and blob storage must be sized and backed up in your environment. DeployOpen does not publish timelines, and most engagements are fixed-scope with scope, deliverables and timelines agreed in writing before work starts. Ask for a timeline that lists each application, each environment and each dependency, and check which items depend on your own change approvals or procurement.
What do we need from our own teams?
Plan for time from platform engineering to provision Kubernetes or a VM, databases, object storage and networking. Application owners must agree which prompts, outputs and metadata may be captured, and security or privacy teams must approve masking and retention rules. Your identity team connects SSO, and your operations team owns on-call and upgrade windows. Confirm named contacts for each of these before the work begins, since delays in approvals usually cost more than the technical work itself.
Can our own engineers run this without an outside partner?
Yes, many teams can deploy Langfuse or Phoenix with their own staff, especially using the Docker Compose path for testing or the Kubernetes path for production. The harder parts are sizing ClickHouse, securing trace data, instrumenting many applications consistently and planning major upgrades. Langfuse says updates within a major version are non-disruptive, but major upgrades such as v3 to v4 require ClickHouse 25.12 or later, PostgreSQL 15 or later and Redis 7.0 or later, plus about 3x ClickHouse disk headroom when backfilling historic data. Assess your team's capacity for those tasks honestly before choosing a build path.
What does an ongoing upgrade and support arrangement cost?
Upgrades and support recur for as long as the platform runs. Major version changes need migration planning, testing and a rollback path, and each new release needs a decision on timing. DeployOpen offers ongoing support, upgrades and continuous testing as retainers, and the scope of each retainer is set in writing. Langfuse's Self-Hosted Enterprise plan includes a support SLA and named support engineer, so compare those terms with any partner retainer. Ask each vendor what response times and upgrade windows they commit to in writing.
Is OpenTelemetry required, and what does it add to the cost?
OpenTelemetry is not required for every integration, but it reduces lock-in and gives you one way to route telemetry. OpenTelemetry is a vendor-agnostic framework for generating and exporting traces, metrics and logs, and it is not a backend, so storage and visualization remain separate tools. The OpenTelemetry Collector, licensed under Apache-2.0, can receive, process and export data and can authenticate incoming connections with extensions such as OIDC. Trace APIs and SDKs are stable in most languages, but log SDK maturity varies. Check the status of each language you use and the stability of each Collector component before you commit.
How DeployOpen scopes and delivers LLM observability requirements
DeployOpen turns LLM observability requirements into a written scope, then plans, implements and hardens self-hosted Langfuse or Phoenix inside your own cloud accounts, data centers or air-gapped networks. Each engagement is fixed-scope, so the applications, storage and evidence are agreed on paper before work starts.
How we scope
- Inventory the applications, agents and frameworks that need tracing, with owners and release paths
- Estimate trace volume and retention periods with your data owners before choosing storage sizes
- Record masking, SSO, project role and license-key decisions in the written scope for sign-off
How we build
- Deploy Langfuse web, worker, PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage in your environment
- Instrument applications through the SDK or OpenTelemetry Collector with masking applied before storage
- Test ingestion under expected load, then verify access controls, backups and restore steps
How we hand over
- Deliver architecture notes, configuration records, runbooks and the requirements traceability matrix
- Document which features rely on a Langfuse Enterprise license key and which are open source
- Train operators on upgrades, including the Langfuse v3 to v4 migration path and ClickHouse headroom
How we support
- Offer ongoing support, upgrades and continuous testing as a retainer after handover
- Review retention, access logs and evaluation outputs on a schedule agreed in the written scope
- Confirm scope changes in writing before any additional instrumentation or storage work begins
