Why LLM observability for healthcare starts with the trace store
Traces are a new store of protected health information. When a clinical or operations team wants to trace an LLM feature that touches patient data, the prompts, retrieved documents and model outputs that land in the trace store can contain PHI just as the source system does, so masking, access and retention must be designed before the first trace is written. Instrumentation that captures everything by default turns the observability platform into an unplanned copy of the patient record. The HIPAA Security Rule (45 CFR Part 164 Subpart C) applies to covered entities and their business associates for all electronic protected health information they create, receive, maintain or transmit (45 CFR 164.302), so a trace store run by the provider or by a vendor on its behalf falls inside its scope. HIPAA has no certification: HHS states that no Security Rule standard requires certifying compliance, and enforcement runs through complaints, compliance reviews and audits. Teams should therefore ask for evidence they can show an assessor, not a badge. Self-hosted Langfuse fits this pattern because the MIT-licensed core covers tracing without a license key, and the data stays in the organization's own PostgreSQL, ClickHouse and object storage. Some controls that matter here, including project-level RBAC, audit logs, data retention policies and server-side data masking, need an Enterprise license key, so that cost belongs in the scope conversation from the start. Confirm the licensing position against the current Langfuse license key documentation before design sign-off. Phoenix is licensed under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service, so it is not a fit if the organization or a partner plans to offer tracing as a service to others. For internal use it can be evaluated, but the masking and access decisions described above still have to be made and tested.
Which healthcare rules shape the trace design
This table maps each rule to what it means for who can see traces, how prompts are masked, where trace data is stored and how incidents are reported. Read each row as a design requirement to confirm with your compliance lead before the trace schema is fixed. The main takeaway is that a HIPAA-covered workload needs access control, audit and reporting terms that are settled before any patient-related prompt reaches Langfuse.
| Rule or standard | What it asks for | Implication for trace design |
|---|---|---|
| HIPAA technical safeguards (45 CFR 164.312) | Unique user identification, emergency access procedure, automatic logoff and encryption, with two of the seven implementation specifications required (unique user identification and emergency access) and the others addressable | Give each person who opens traces an individual account, define a break-glass path for urgent access, and decide how encryption and session timeouts are handled in Langfuse and its storage |
| HIPAA business associate contract (45 CFR 164.314) | The business associate must comply with the Security Rule, flow the same terms to subcontractors that handle ePHI, and report security incidents, including breaches of unsecured PHI | Confirm that the Langfuse hosting provider, any implementation partner and any storage vendor are covered by contracts, and that trace-level incidents have a reporting path to the covered entity |
| HIPAA Security Rule scope (45 CFR 164.302) | Applies to all electronic protected health information that a covered entity or business associate creates, receives, maintains or transmits | Treat any trace that contains PHI as ePHI, including traces that carry diagnosis text, patient identifiers or clinical notes passed to a model |
| EU AI Act record-keeping (Articles 12(1) and 19(1)) | High-risk systems must allow automatic event logging over the system's lifetime, and providers must keep those logs for at least six months unless other law provides otherwise | Set trace retention so logs survive the six-month minimum, and confirm whether retention and log deletion must be reconciled with healthcare record rules in your jurisdiction |
| GDPR Articles 32 and 34 (security and breach notice) | The CNIL fined Hopital Prive de la Loire EUR 500,000 on 3 September 2026 for external access without VPN or multi-factor authentication and for access control that ignored the care team's need to know | Require multi-factor authentication for any remote access to trace data, restrict project roles to the care team that needs them, and plan breach notice for trace stores |
| NIS2 health sector scope (Annex I) | Lists healthcare providers, EU reference laboratories, medical device manufacturers and related health entities as sectors of high criticality | Confirm whether your organization falls within NIS2 scope, and if so, include Langfuse and its database and storage services in the incident and supplier risk process |
| Storage location and masking | Regulators assess where patient-derived data is stored and who can reach it, and the cited fine turned on external access paths | Keep ClickHouse, PostgreSQL and object storage in approved locations, and mask identifiers in the SDK or OpenTelemetry Collector before data leaves the application |
| Audit logs on trace access | Audit logs and access reviews are needed to show who viewed or exported patient-related traces | Confirm which audit log and retention policy features need a Langfuse Enterprise license key and whether your sign-off requires them before go-live |
Which open-source platforms fit a PHI-handling LLM stack
A PHI-handling LLM stack needs a tracing backend, a collector that can redact data before it is stored, and a gateway that records model calls. The platforms below cover those roles, but their license terms and feature gates differ, and those differences decide which controls you can run in your own environment.
Self-hosted Langfuse
Langfuse's core is MIT licensed, but the ee directories fall under a separate enterprise license. Several controls auditors ask about sit behind an Enterprise license key: project-level RBAC, audit logs, data retention policies and server-side data masking. Before you commit, map each required control to the license tier and confirm with counsel which tier your deployment needs. Ask the partner to show the key-gated features working in your environment, not only in documentation.
Arize Phoenix under ELv2
Phoenix is licensed under the Elastic License 2.0, which permits self-hosting inside your organization but prohibits offering the software to third parties as a hosted or managed service. For a healthcare provider running Phoenix for its own staff, that is usually a workable position. It becomes a concern if a business associate or vendor operates the tracing platform for you as a service, so confirm the operating model and contract terms with counsel before selecting it.
OpenTelemetry Collector for redaction
The OpenTelemetry Collector is Apache-2.0 licensed and sits between your applications and the trace backend. It can receive, process and export telemetry, so you can drop or mask prompt and output fields before anything reaches storage. Authenticator extensions can also require OIDC authentication on incoming connections. Treat the collector configuration as a security control: test that masked fields never leave the collector, and review its component stability levels for the processors you rely on.
LiteLLM as a logging gateway
LiteLLM sits between applications and model providers and logs calls in one place, with cost tracking and guardrails. For PHI workloads, it centralizes which model received which request and under which key, which simplifies audit trails. Its enterprise directory uses separate terms, so confirm which features you need are in the MIT-licensed part. Check that request logging can be configured to exclude or mask payload content before it is written.
Licensing questions for counsel
Open-source licenses are not the only decision. Whether a feature gate, a hosted-service restriction or an enterprise directory affects your deployment depends on who operates the stack and who has access to PHI. Put these questions to counsel in writing: which Langfuse controls your compliance program depends on, whether your operating model is a hosted service under the Elastic License 2.0, and whether the enterprise license terms fit your procurement rules.
How to mask PHI before traces reach storage
Masking should happen before trace data leaves the application or the collection layer, not after it lands in storage. Langfuse's server-side data masking is one of the features that requires an Enterprise license key on self-hosted deployments, so an organization relying only on the open source edition should plan to redact in the application SDK or in the OpenTelemetry Collector. Either point sits in front of the self-hosted ClickHouse and S3-compatible blob storage, which means raw patient identifiers never reach the trace database or its backups. The decision to make first is which layer owns the redaction rules, and the answer should be written down before any instrumentation starts.
Identifiers appear in three places: the user prompt, the retrieved text (for example chart notes or discharge summaries pulled by a RAG step) and the model output, which can repeat or infer names, dates of birth, medical record numbers and addresses. A rule set usually combines pattern matching for structured identifiers with named-entity detection for free text, applied to each of these three fields. The OpenTelemetry Collector's processors can drop or transform span attributes before export, and the same logic can run in the SDK so the raw value is never serialized. Illustrative example: a rule that replaces anything matching a medical record number format with a fixed token and records only the token type. Treat the rule set as configuration under version control, not as code written once.
Test masking with synthetic PHI before any real patient data touches the pipeline. Build a test corpus of invented patients, with realistic-looking but fictitious identifiers, run it through the full path, and query the stored traces to confirm nothing identifiable survives in prompts, outputs, retrieved context or metadata fields. Redaction rules also need clinical and privacy review: a clinician can spot clinically meaningful content that a pattern would strip out, and a privacy officer can confirm the rules match what the organization has documented as PHI. Ask your implementation partner to show the test corpus, the expected results and the change process for updating rules, and confirm with your compliance team how masked traces are classified under your own risk assessment.
How a healthcare rollout of self-hosted tracing proceeds
The sequence moves from classifying what data reaches traces and confirming which parties handle it, through masking tests on synthetic records, deployment and access controls, to a documented handover that the compliance and clinical owners can review. Each phase ends with a named artifact so the organization can see what it has before moving on.
Classify trace data and scope business associates
The rollout starts by mapping every application, prompt template, tool call and retrieved document that could place electronic protected health information in a trace. The privacy officer, security lead and application owners agree which fields are PHI, which are merely sensitive, and which are safe to store. In parallel, the team confirms which parties would create, receive, maintain or transmit ePHI, including the hosting provider and any support firm, so business associate terms under 45 CFR 164.314 can be checked before any real data flows. Output: a signed data inventory and a list of parties with the contract status of each.
Design masking rules and test on synthetic PHI
Masking is designed on the instrumentation side, in the application SDK or the OpenTelemetry Collector, so identifiers are removed before they leave the workload. The team writes rules for names, medical record numbers, dates of birth, addresses and free-text clinical notes, then tests them with synthetic patient records that mimic the real formats. Reviewers check the stored traces, not the code, to confirm nothing identifiable reaches ClickHouse or blob storage. Output: a masking specification and a test report showing which synthetic identifiers were caught and which were missed, with fixes applied.
Deploy in the organization's environment
The Langfuse stack is deployed inside the organization's own cloud account or data center, with PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage. Docker Compose on a single VM is used only for testing because it has no high availability. Production runs on Kubernetes using the Helm or Terraform-based approaches Langfuse documents, with resource sizing taken from the scaling guide and checked against expected trace volume. Output: a production environment with network paths, encryption and backups documented, and a load test record showing the sizing holds.
Configure SSO, project roles and audit logs
Single sign-on is connected to the organization's identity provider so access follows the same joiner and leaver process as other clinical systems. Project-level roles are set so that only staff with a need to know can view traces for a given application. Several of these controls, including project RBAC, audit logs, server-side data masking and the SCIM provisioning API, need a Langfuse Enterprise license key, so the license decision is made before this step. Output: an access matrix mapping roles to projects, a test showing a user without rights cannot open a trace, and a confirmed audit log export.
Set and enforce retention policies
Retention periods are set per project, based on the clinical, legal and AI Act logging needs identified in phase one. Where the system is high-risk under the EU AI Act, automatically generated logs must be kept for at least six months unless other law provides otherwise, so that minimum is a floor, not a target. The team verifies that expired traces are deleted from every store, including backups and object storage, and that the deletion process is documented. Output: a retention schedule approved by the data owner and a deletion test on synthetic records.
Hand over operations and document evidence
Operational handover covers upgrade procedures, since Langfuse applies database migrations at startup for minor updates and major versions follow dedicated migration guides, backup and restore tests, alerting on ingestion failures, and the incident path for a suspected PHI exposure in a trace. The compliance team receives an evidence pack mapping controls to the HIPAA Security Rule standards they are assessing against. DeployOpen prepares this evidence, but the covered entity and its auditors decide what it demonstrates. Output: signed runbooks, a restore test record and a handover meeting with named owners for each control.
What evidence a healthcare auditor or regulator will ask for
What access logs and role assignments will an auditor ask to see for trace data?
Auditors usually ask who can view traces that contain patient information, how those roles were granted, and whether access matched the care team's need to know. In self-hosted Langfuse, project-level RBAC roles are an Enterprise license feature, so confirm early whether your design depends on them. Prepare a current list of users and roles per project, the approval record for each grant, and periodic access reviews with removals documented. The HIPAA Security Rule's unique user identification and automatic logoff specifications in 45 CFR 164.312 are the controls these records most often support.
What masking test records prove that PHI stays out of stored traces?
Regulators and auditors look for evidence that masking works, not only a description of it. Keep test cases that send known identifiers such as names, record numbers and dates through the instrumented application, then show the stored trace with those values removed or replaced. Record the masking rules version, the date of each test, the tester, and the result for every failed case with its fix. Where masking runs in the SDK or OpenTelemetry Collector, keep the configuration files under version control so each change is traceable.
What retention settings will a compliance reviewer expect to see documented?
A reviewer will expect a written retention period for each category of trace data, the configured setting that enforces it, and proof that deletion runs. HIPAA does not set a retention period for trace data in the Security Rule text, so the period comes from your own policy and any other law that applies. Data retention policies in self-hosted Langfuse are an Enterprise license feature. If you do not hold that license, document the manual or scheduled deletion procedure for ClickHouse, PostgreSQL and object storage instead, and confirm it with your counsel.
Can audit log exports from the observability platform satisfy an audit trail request?
Audit logs are a self-hosted Langfuse feature that requires an Enterprise license key, so check whether your edition includes them before you promise the evidence. Where they are available, export them on a schedule to storage you control, with a retention period that matches your policy. Test that an export can be read by someone outside the engineering team. If audit logs are not licensed, record the equivalent evidence from your identity provider, reverse proxy and database logs, and confirm with your assessor that it meets the scope of the request.
What business associate contract terms should cover a vendor that can see trace data?
Any vendor that creates, receives, maintains or transmits ePHI on your behalf is a business associate, and 45 CFR 164.314(a)(2)(i) requires a written contract with specific terms. The contract must require compliance with the Security Rule, flow the same terms down to subcontractors that handle ePHI, and require reporting of security incidents, including breaches of unsecured PHI. Ask any implementation partner or hosting provider whether they will access trace data, and document the answer. Your legal team should confirm whether a contract is needed for each party, including the company that provides support.
Is there a HIPAA certification for an LLM observability deployment, and what can a HITRUST assessment add?
No. HHS states in its FAQ from 2003 that no Security Rule standard requires an entity to certify compliance, and that private certifications do not prevent HHS from later finding a violation. The HHS Office for Civil Rights enforces the rule through complaints, compliance reviews and audits. A HITRUST assessment is performed by an independent assessor under the HITRUST CSF, and its results are a separate, voluntary report. DeployOpen does not issue HIPAA certifications or HITRUST assessments, so ask your assessor which evidence they will accept.
What incident response procedures should cover a leak of patient data in trace storage?
Your procedure should define how a suspected exposure of PHI in traces is detected, who is notified, how access is cut off, and how the covered entity and any business associate report it. The business associate contract must require reporting of security incidents, including breaches of unsecured PHI, so the timing and channel should be written into that contract. Prepare runbooks that include revoking project keys, rotating credentials, restoring storage from backup, purging affected traces and preserving logs for investigation. Test the runbook with a tabletop exercise and keep the notes as evidence.
What to ask an implementation partner before scoping
Start with the storage map. Ask the partner to show, component by component, where prompts, outputs, metadata and blob payloads land in a self-hosted Langfuse deployment: PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage. Each of these holds a copy of something a clinician or patient may have typed, so the answer should name the hosting account or data center, the encryption and key management for each store, and which identities can read them. Ask for a written data flow diagram before any instrumentation starts. A partner that cannot draw it is not ready to scope a PHI workload.
Ask how masking is tested and who signs it off. Client-side masking in the SDK or the OpenTelemetry Collector is the control most likely to fail silently, so ask for the test set of synthetic patient records the partner will run through the pipeline, the pass criteria, and the named clinical or privacy reviewer who accepts the result. Then ask which Langfuse features need an Enterprise license key, since server-side data masking, data retention policies, audit logs and project-level RBAC roles sit in that tier. Ask for the cost of that license as a separate line in the quote, and ask whether the partner will document what happens to the trace data if the license is not purchased.
Ask how ClickHouse and blob storage are backed up and restored, and ask for a timed restore test, not a description of the backup job. Confirm which copies are encrypted and where they live, how a deletion request or retention expiry reaches backups, and who approves a restore. Ask how access reviews and audit logs are produced: which accounts hold administrator rights, how often reviews run, and where the evidence is kept. Finally, if the partner's staff can reach PHI during setup, troubleshooting or support, ask for its business associate agreement under 45 CFR 164.314 and how it flows the same terms to its own subcontractors. The contract should be signed before access, not after.
What does healthcare LLM tracing cost and take to implement?
What are the main cost components of self-hosted LLM tracing for healthcare?
The cost has four main parts: the infrastructure your organization runs, the implementation work, the masking and testing work, and any Langfuse Enterprise license. The infrastructure covers compute, storage and backups for PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible object storage, which the self-hosting docs list as the core dependencies. Implementation covers deployment, SDK or OpenTelemetry instrumentation of each LLM feature, and identity integration. Ask any partner to price each component separately so you can see which parts recur.
Which parts of healthcare LLM tracing cost money every year?
Infrastructure is the main recurring cost, since servers, cloud accounts, storage growth and backups run continuously. Upgrades are the second recurring item: Langfuse applies database migrations at startup within a major version, while major upgrades such as v3 to v4 follow dedicated migration guides that require newer ClickHouse, PostgreSQL and Redis versions. Enterprise license fees, if you need them, recur under the vendor's terms. Masking rules and evaluation sets also need periodic review as prompts and data types change.
What drives the cost of masking PHI in traces up or down?
Cost rises with the number of LLM features, the variety of data they send and the number of places where traces are created. Masking applied in the SDK or the OpenTelemetry Collector needs development in each application, plus test cases that prove identifiers are removed before storage. Costs fall when features share one instrumentation library and when the data types are limited and well documented. Ask a partner to list the identifier types they will test for and how they will prove masking worked on sample traces.
Can our internal team run healthcare LLM tracing without a partner?
Yes, if you have engineers who can operate PostgreSQL, ClickHouse, Redis or Valkey and object storage, run Kubernetes or Docker Compose, and maintain application instrumentation. The Langfuse documentation covers Docker Compose for testing and Helm or Terraform deployments for production, so the path is well documented. The harder parts are sustained: keeping masking rules correct, handling access reviews, and planning major upgrades on a schedule your clinical and security teams accept. Confirm you can staff those duties before you choose a build approach.
What does our team need to provide for an implementation engagement?
Plan to provide a named technical owner, access to the environments where the LLM features run, a list of the applications and workflows to instrument, and a privacy or clinical data owner who can approve masking rules. Your security team should supply identity provider details for SSO, network and hosting constraints, and the retention periods that apply to trace data. Security and legal reviewers will also need time to approve the business associate terms that cover any vendor who handles ePHI.
How is the scope of a healthcare tracing engagement agreed?
DeployOpen's engagements are typically fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. A written scope for this topic should list the applications to instrument, the masking rules and how they will be tested, the Langfuse features that depend on a license key, the environments, the backup approach and the handover documentation. Any item left out of the written scope is a change to be priced separately. Request the written scope before you compare quotes, since differing scopes make prices incomparable.
Which Langfuse features may need an Enterprise license for healthcare use?
Self-hosted Langfuse lists several features that need an Enterprise license key, including project-level RBAC roles, data retention policies, audit logs, server-side data masking, and SCIM with the org management API. Because masking and audit logging matter for patient data, these features often shape the license decision. Confirm the current license terms and feature list directly with Langfuse before you finalize a budget, because the pricing page describes the Self-Hosted Enterprise plan as custom-priced.
How DeployOpen scopes and supports PHI-aware tracing
DeployOpen plans, implements and documents self-hosted Langfuse inside the client's own environment, whether that is their cloud accounts, data center or a restricted network. Scoping starts with an inventory of which LLM features touch patient data, which applications and frameworks will send traces, and which trace fields may contain PHI. From that inventory the work covers the deployment of Langfuse Web, Langfuse Worker, PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage, the placement of masking in the SDK or OpenTelemetry Collector, the configuration of project roles and SSO, and the retention settings. Each control is written down with its configuration, owner and test, so the client holds the design as well as the running system. Most engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts, and the quote follows from that written scope.
DeployOpen also runs security testing against the masking and access paths, because a control that exists on paper is not proof that PHI stays out of trace storage. Testing targets the places where data leaks in practice: prompts and retrieved documents that reach traces unmasked, API keys or service accounts with broader project access than intended, and audit trails that do not record who viewed a trace. Findings are fixed in the configuration and retested. DeployOpen prepares systems, controls and evidence for assessments such as HIPAA reviews or HITRUST work, including access records, configuration baselines and test results. It does not issue audit reports, attestations or certifications. Clients should confirm with their auditor and legal counsel which of their own obligations the evidence is meant to support, since HIPAA has no certification.
Ongoing work is available as a retainer covering Langfuse upgrades, security patches, evidence refreshes and periodic retesting of the masking rules as the applications that feed traces change. Before signing, ask the partner which upgrade path they follow for major versions, how they handle the ClickHouse headroom and migration steps in the Langfuse v3 to v4 guide, and who on their side is accountable for a masking failure found after go-live. Ask for the scope, deliverables and responsibilities in writing so the client can compare them with the terms of its own business associate agreements.
