How LLM observability supports ISO 42001 controls for AI operation and logs
If your organization runs LLM features and must show that its AI management system is operated and monitored, LLM tracing can supply much of the operational and event-log evidence behind Annex A controls A.6.2.6 (AI system operation and monitoring) and A.6.2.8 (AI system recording of event logs). ISO/IEC 42001 is a management system standard that does not certify anything. External certification bodies make that decision, so the job is to build a system that produces evidence those bodies can test. A.6.2.6 asks you to define and document what the ongoing operation of each AI system needs, covering at least system and performance monitoring, repairs, updates and support. Traces, latency and error dashboards, and cost records in a self-hosted Langfuse deployment can answer the monitoring part, while your runbooks and change records answer the rest.
Annex A controls are not all mandatory. Under clause 6.1.3, you document which controls are necessary and justify including or excluding each one in a statement of applicability. Your statement should show why A.6.2.8 applies to each LLM feature and at which life cycle phases event logging is enabled, with an absolute minimum of when the system is in use. Confirm this scope with your certification body early, since it decides which traces you must keep and for how long. DeployOpen plans and configures the tracing stack, maps its outputs to the controls you select, and prepares the evidence. DeployOpen does not issue audit reports, attestations or certifications, and the statement of applicability remains your organization's decision.
Mapping ISO 42001 controls to what LLM tracing must do
This table maps the ISO/IEC 42001 Annex A controls on AI system operation, monitoring and event logs, plus the statement of applicability, to what the tracing function must capture and how to configure Langfuse or Phoenix to produce that evidence. Read each row as a control, the duty it creates for the tracing layer, and the configuration an assessor will ask to see.
| Requirement | What the tracing function must do | How to configure Langfuse or Phoenix |
|---|---|---|
| A.6.2.6 AI system operation and monitoring | Define and document the ongoing operation of each LLM application, including monitoring of system and performance, plus the owners for repairs, updates and support | Instrument each application with the Langfuse SDK or OpenTelemetry, tag traces by application, version and environment, and set up dashboards and alerts on latency, errors and cost per trace |
| A.6.2.6 performance evidence | Record quality signals over time so performance changes can be traced to a prompt, model or release | Attach scores to traces through evaluators or user feedback, store the prompt version on each trace, and export score trends for review meetings |
| A.6.2.8 AI system recording of event logs | Decide which life cycle phases need logging and make sure logs exist at least whenever the system is in use | Enable trace capture in production and confirm the collector or SDK sends events to the self-hosted Langfuse Web and Worker containers without dropping data during outages |
| A.6.2.8 log content and integrity | Capture enough detail to reconstruct a session, including inputs, outputs, model, timestamps and the identity of the caller | Record model name, token counts, cost fields, user or service identifiers and timestamps on each observation, and test that a sample trace can be rebuilt end to end |
| A.6.2.8 retention of logs | Keep event logs for the period set by the organization's policy and any applicable law, and delete them when that period ends | Set retention in the self-hosted deployment and confirm whether data retention policies need the Langfuse Enterprise license key, since the feature list in the license docs includes them |
| Access to trace data | Limit who can view prompts and outputs, and record who changed settings or exported data | Use project-level RBAC roles and confirm audit logs and SCIM provisioning are available in your license tier, because the license key docs list these as Enterprise features |
| Clause 6.1.3 statement of applicability | Justify whether each Annex A control in the applicable scope is included or excluded, with the tracing evidence linked to each included control | Map each control to the trace view, dashboard or export that proves it, and record in the statement which Langfuse or Phoenix features are used and which are not in place |
| Phoenix licensing check | Confirm the tracing platform fits the organization's own use, not a service offered to third parties | Phoenix is under the Elastic License 2.0, which prohibits providing it as a hosted or managed service, so confirm the chosen deployment model with counsel before selection |
Which self-hosted platform fits which tracing situation
Langfuse is the option most teams evaluate first for storing traces, prompts, outputs and evaluation scores in their own storage. Its core features and APIs are available under the MIT license in the open source version, and self-hosted deployments use the same codebase as Langfuse Cloud. The platform needs PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible blob storage, so the evidence it produces is only as controlled as the infrastructure beneath it.
- Core tracing under MIT: Traces, observations and evaluations are available without a license key in the open source version.
- Enterprise features behind a key: Project-level RBAC roles, data retention policies, audit logs, server-side data masking and the SCIM-based org management API need an Enterprise license key.
- Separate license directories: Files under the ee directories fall under a separate enterprise license, so check which code paths your deployment touches.
- Operational footprint: Web, Worker, PostgreSQL, ClickHouse and Redis or Valkey each need sizing, backup and upgrade planning.
- Upgrade discipline: Major upgrades follow dedicated migration guides and require newer ClickHouse, PostgreSQL and Redis versions.
Confirm with your auditor which Enterprise features are required as evidence before you choose the open source tier, because audit logs and retention controls are central to showing how operation and event records are governed.
Arize Phoenix is an LLM tracing and evaluation tool that can serve teams who want a lighter evaluation workflow. Its license is the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service. Internal use by your own staff on your own infrastructure is a different situation from offering monitoring to customers, so the license question must be settled before Phoenix is put into a control narrative.
- Internal deployment: Running Phoenix for your own teams on infrastructure you control is the case to confirm against the license text.
- Hosted or managed offering: Providing Phoenix to clients or as part of a managed service is the restriction that applies.
- Evaluation focus: Phoenix is strong in tracing and evaluation experiments, so compare the features needed for logging and retention with Langfuse.
- Retention and access features: Confirm which access controls, retention settings and audit records your version provides rather than assuming them.
- Vendor relationship: DeployOpen is not a Phoenix reseller or partner, so confirm licensing questions with Arize or counsel.
Ask your legal team to read the Phoenix LICENSE file against your deployment model, and document the decision in your AI system records.
The OpenTelemetry Collector is not a trace store. It receives, processes and exports telemetry, and it sits between your applications and whichever backend you choose. Because the Collector is Apache-2.0 licensed and vendor-agnostic, it lets you change backends later without rewriting instrumentation. It is useful wherever you need consistent handling of trace data before it reaches storage, including masking and routing.
- Agent or gateway deployment: Run it next to services as an agent, or as a central gateway that accepts data from many sources.
- Authentication on receivers: Authenticator extensions such as OIDC can authenticate incoming connections.
- Outgoing credentials: The oauth2client extension can add credentials to exporter requests.
- Signed releases: Contrib distributions ship with signed artifacts, SHA256 checksums and SBOM files, which supports supply chain review.
- Mixed component stability: Core components have different stability levels, so check each component's README before relying on it.
Document which Collector components and extensions are in your build, and pin versions as part of your change records.
LiteLLM is an AI gateway that sits between applications and model providers. It calls more than 100 LLM APIs in OpenAI format and provides cost tracking, guardrails, load balancing and logging. This makes it a natural place to record model calls once, regardless of which application makes them. It is MIT licensed except for the enterprise directory, so the features that fall under that directory must be checked before adoption.
- Central call logging: Requests from many applications pass through one point where logs can be recorded.
- Cost tracking: Usage and spend per model can be attributed for budget and operation reviews.
- Enterprise directory: Features in the enterprise directory carry a different license, so confirm what your edition uses.
- Complementary role: The gateway records calls, while Langfuse or another backend stores traces and evaluations for review.
- Access to logs: Confirm who can read gateway logs and where they are written.
Decide which system is the authoritative record for each event type, so auditors are not sent to two sources with different retention.
Sizing and deployment decisions that shape the evidence
The way Langfuse is deployed and upgraded determines how long traces are kept, how reliably they are written and whether the evidence survives a platform change. These decisions should be made before the first audit request, not after.
Single VM for testing only
Langfuse documents Docker Compose on a single VM as a testing setup with no high availability. Traces written during a pilot may still be needed as operating evidence, so a test VM is a poor place to hold records you may later have to produce. Confirm with the team whether pilot data will be migrated, discarded or kept as a documented record before the production build begins.
Kubernetes for production
Langfuse documents Kubernetes deployments with Helm or Terraform on AWS, Azure and GCP for production. Using these standard deployment paths makes backups, restores, network policy and access control repeatable, which is what an assessor will ask to see. Ask the implementation partner to show how the chart or module values are version-controlled and who approves changes to them.
Minimum resources in the scaling guide
The scaling guide lists minimum resources for web, worker, PostgreSQL, Redis or Valkey and ClickHouse, with ClickHouse needing the most memory for larger deployments. Undersized storage can drop or delay traces, which leaves gaps in the monitoring record for A.6.2.6 and the event logs for A.6.2.8. Ask for a sizing estimate based on your expected trace volume and retention period, not the minimums.
Scaling workers on CPU load
Langfuse recommends scaling the worker containers on CPU load, since the worker processes events asynchronously. If the worker falls behind, traces arrive late or not at all, and the record of an AI system's operation becomes incomplete for the period in question. Set alerts on worker CPU and queue lag, and keep the alert history as part of the monitoring evidence.
Splitting web under heavy ingestion
Under heavy ingestion, Langfuse recommends splitting langfuse-web into separate ingestion and user interface deployments. This keeps a busy evaluation or dashboard session from slowing the path that records events. Ask the partner how ingestion capacity was tested against your peak request rate, and where that test is recorded.
Upgrading from v3 to v4
The v3 to v4 guide requires ClickHouse 25.12 or later, PostgreSQL 15 or later and Redis 7.0 or later, and moves to an observations-first data model. Backfilling historic data needs about 3x ClickHouse disk headroom, so a major upgrade must be planned with storage in mind. Confirm the rollback plan and the retention of pre-upgrade traces before the migration starts.
Evidence auditors ask for and which component produces it
This table maps the records an assessor typically asks for during an ISO/IEC 42001 review of AI system operation and monitoring to the component that produces each one. Read the last two columns together: the producing component shows where the evidence originates, and the Enterprise key column shows where a Langfuse feature may need a paid license. The main takeaway is that much of the evidence comes from the organization's own procedures and change records, not from the tracing tool alone.
| Evidence | Producing component | Enterprise key needed |
|---|---|---|
| Documented operation and monitoring procedure | The organization's management system documents, approved by the accountable owner | Not applicable |
| Trace records with timestamps and model versions | Langfuse, populated by instrumented applications or OpenTelemetry Collector | No |
| Performance and evaluation scores | Langfuse evaluations, with reviewer decisions recorded in the organization's own process | No |
| Cost and latency records | Langfuse trace metadata, with budgets and alerts set in the organization's monitoring tools | No |
| Repair and update change records tied to upgrades | The organization's ticketing or change system, linked to Langfuse release and migration records | Not applicable |
| Retention settings and deletion logs | Langfuse data retention policies and audit logs, plus storage lifecycle logs from the object store | Yes |
| Access reviews and audit logs | Langfuse audit logs and project-level roles, with SSO identity records from the identity provider | Yes |
| Collector ingestion and authentication records | OpenTelemetry Collector pipeline configuration and authenticator extension settings | No |
What tracing alone does not satisfy under ISO 42001
A self-hosted tracing tool records evidence for specific Annex A controls, but it does not make an AI management system exist. The cards below cover the surrounding obligations that a trace store cannot supply, and the points where the certification path is still unsettled.
The management system around the tool
Clause 4 to 10 of ISO/IEC 42001 require a documented AI management system: policies, assigned roles, objectives and a risk process. Langfuse can show what a model did, but it cannot decide who owns an AI system, who approves a prompt change or who acts on a drop in evaluation scores. Map each trace review to a named role and a documented procedure. Risk assessment can follow ISO/IEC 23894:2023 on AI risk management, and that assessment should name the risks that tracing is expected to detect.
The statement of applicability
Clause 6.1.3 requires a statement of applicability that lists the Annex A controls the organization needs and justifies each inclusion or exclusion. Tracing evidence supports A.6.2.6 and A.6.2.8, but the statement must also explain why other controls are excluded or handled elsewhere. Auditors will check that the justification matches what the tracing setup actually records. Confirm the wording with your certification body before you finalize it.
Impact assessment under ISO/IEC 42005
ISO/IEC 42005:2025 guides the assessment of impacts an AI system may have on individuals, groups and society. Traces show what the system produced in practice, which is useful input for that assessment, but traces do not identify the harms in advance. Use the impact assessment to decide which fields, outputs and users must be monitored, and set alerts or reviews around those findings. Record the link between each impact finding and the monitoring that follows.
Personal data inside traces
Prompts, retrieved documents and outputs often contain personal data, so the trace store becomes a personal data repository in its own right. Before tracing goes live, decide what is masked in the SDK or OpenTelemetry Collector, who may view raw traces and how long they are kept. Several controls for this, including server-side masking and retention policies, are Langfuse enterprise features, so confirm licensing before you rely on them. Data quality of logged inputs also needs a documented check.
No accredited certification path yet
ISO does not certify organizations. Certification bodies do, and ISO/IEC 42006:2025 sets additional requirements for bodies certifying AI management systems. As of the latest Global ACI material, ISO/IEC 42001 is not listed as an endorsed sub-scope of the Global ACI recognition arrangement, unlike ISO/IEC 27001. Ask any certification body whether its accreditation covers ISO/IEC 42001 before you plan around a certificate. Tracing can prepare evidence, but it does not settle which accreditation applies.
Separate EU AI Act quality management
The European Commission states that ISO/IEC 42001 helps set up an AI management system but does not match the quality management system required by the EU AI Act. The harmonized standard EN 18286:2026 was cited in the Official Journal on 14 October 2026. If you place high-risk AI systems on the EU market, map your tracing evidence to both frameworks separately. Confirm with legal counsel which obligations apply to your system.
How much effort and cost does ISO 42001 log evidence take, and what do we need to do?
What drives the cost of ISO 42001 log evidence from LLM tracing?
The cost is driven mainly by how many applications you instrument, how much trace data they produce, and how much operational support the platform needs. Each application needs tracing added to its code or gateway, which is one-time engineering work. Self-hosted Langfuse runs Langfuse Web, Langfuse Worker, PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible blob storage, so storage sizing and infrastructure run recurring costs. Langfuse's self-hosting documentation lists the minimum resources for each component, and a quote should convert those into your own hosting costs. Ask the partner to price instrumentation, infrastructure, backups, upgrades and any Langfuse Enterprise license separately, since the free open source tier and the enterprise tier differ in features.
Which Langfuse features need an Enterprise license for ISO 42001 evidence?
Several features that help with access control and log governance require an Enterprise license key on a self-hosted instance. These include project-level RBAC roles, protected prompt labels, data retention policies, audit logs, server-side data masking and the org management API with SCIM. Core tracing, evaluation and APIs are available in the MIT-licensed open source version. Whether you need the enterprise tier depends on how you plan to show who accessed traces and how long records are kept, so confirm this with your compliance lead before the quote. Langfuse's license key documentation lists the full set.
Can our team run ISO 42001 tracing without an implementation partner?
A team with container, database and Kubernetes skills can run self-hosted Langfuse, and Langfuse documents Docker Compose for testing and Helm or Terraform deployments for production. The harder parts are usually outside the platform: instrumenting each application consistently, defining which events are recorded under the operation and monitoring control and the event log control, masking sensitive content, documenting the procedures, and keeping the platform upgraded. Langfuse notes that major upgrades follow dedicated migration guides and require newer ClickHouse, PostgreSQL and Redis versions. Assess your staff against those tasks, and budget for ongoing operations regardless of who builds the first version.
How long does an ISO 42001 tracing rollout take and what does it depend on?
No duration is supplied for this work, so get a timeline in writing from any partner before committing. The main variables are the number of AI systems in scope, whether they already emit OpenTelemetry traces, how many environments must be covered, how much data masking and access design is needed, and whether the infrastructure is already available. A test deployment on a single virtual machine is documented by Langfuse as having no high availability, so production planning needs a separate architecture step. Ask for milestones tied to scope agreed in writing.
Is Langfuse Cloud a simpler alternative to self-hosting for ISO 42001?
Langfuse Cloud removes infrastructure work, but it moves prompts, outputs and traces into a service that the vendor operates, so where the records sit becomes a question for your data protection and sovereignty review. Self-hosting keeps the logs in your own storage, which matters when you must show where event logs are retained, who can read them and how long they last. ISO/IEC 42001 does not require a particular hosting model, so the choice is a business and risk decision. Weigh the operating effort of self-hosting against those evidence and data residency needs, and document the decision in your statement of applicability.
Does LLM tracing mean we will pass ISO 42001 certification?
No. Tracing produces evidence that supports controls such as A.6.2.6 on AI system operation and monitoring and A.6.2.8 on recording event logs, but ISO does not certify organizations and external certification bodies make that decision. Those bodies assess your whole AI management system, including policy, risk assessment, roles and the statement of applicability under clause 6.1.3. DeployOpen prepares systems, controls and evidence for compliance assessments but does not issue audit reports or certifications. Ask a certification body which records they expect to see before you scope the tracing work.
Does the team need ongoing support after the ISO 42001 logging go live?
Yes, the evidence needs to keep running. Clause 9.1 covers monitoring of the management system, while A.6.2.6 covers monitoring of each AI system, so trace and alert reviews must continue after launch. Langfuse says minor updates within a major version are non-disruptive, but major upgrades require migration work and ClickHouse capacity planning. DeployOpen offers ongoing support, upgrades and continuous testing as retainers. Ask any partner which operational tasks are covered by the retainer and which remain with your team.
How a fixed-scope rollout produces the evidence file
The sequence below turns a scope agreement into a self-hosted tracing stack, documented monitoring and logging procedures and an evidence pack that an independent auditor can sample. Each step closes with a check the client can verify, so the file is built during delivery rather than reconstructed before an assessment.
Confirm AI systems in scope and the statement of applicability
The client and DeployOpen list every LLM application that falls within the AI management system, with its owner, purpose, users and the data its prompts and outputs contain. The team then maps the relevant Annex A controls, including A.6.2.6 on operation and monitoring and A.6.2.8 on recording event logs, into the statement of applicability required by clause 6.1.3, and records why any control is excluded. The step closes when the business owner signs the scope and the statement of applicability, so every later artifact traces back to a named system and a justified control set.
Deploy the self-hosted tracing stack
DeployOpen installs Langfuse in the client's own cloud account, data center or air-gapped network, with its web and worker containers, PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible blob storage. Sizing follows the Langfuse scaling guidance and the client's expected trace volume, and the team confirms the license tier needed for the access, masking and retention features the evidence depends on. The step closes with a health check of every component and a restore test of the database and blob storage, so the storage the logs depend on is known to work before any application writes to it.
Instrument applications and gateways
Engineers connect each in-scope application to the stack, using the Langfuse SDKs or OpenTelemetry instrumentation routed through an OpenTelemetry Collector where the client wants a central point for filtering and authentication. Traces capture model calls, inputs, outputs, versions of prompts and models, latency, token counts and cost. The step closes when a test request from each system appears in Langfuse with the expected fields, and the team records which fields are masked before storage.
Define monitoring and logging procedures
The client and DeployOpen write the procedures that A.6.2.6 asks for: what is monitored for each system (performance, errors, cost, quality scores), who reviews the signals, how alerts are handled, and how repairs, updates and support are tracked for the ongoing operation of the system. A parallel procedure states the phases of the life cycle in which event logging is enabled, which must include at least the period when the system is in use, as A.6.2.8 requires. The step closes when the procedures are approved by the named owners and linked from the statement of applicability.
Set retention and access rules
The client decides how long traces are kept, based on its own legal and policy obligations, and DeployOpen configures the retention and project-level access roles in Langfuse, including which staff may view traces that contain personal data. Where the EU AI Act applies, the client confirms the retention period against Article 19(1), which sets a minimum of six months unless other law provides otherwise. The step closes with a documented retention schedule and an access matrix, plus a test showing that a user without the right role cannot open a restricted project.
Run a first review cycle
The team runs a documented review of a sample of traces against the procedures: whether each in-scope system is recording events, whether monitoring alerts were acted on, and whether any gaps exist in masking, retention or access. Findings are logged with an owner and a fix date agreed with the client. The step closes when the review record is signed and each finding is either closed or accepted with a documented reason, which is the operating evidence an auditor will ask to see.
Produce the evidence pack for the auditor
DeployOpen assembles the evidence pack: the scope and statement of applicability, the deployment and configuration records, instrumentation test results, the monitoring and logging procedures, the retention and access records, and the review cycle output. The pack is organized by control ID so an independent certification body can sample it. The step closes when the client confirms the pack is complete and stored in its own repository. Certification itself remains with an accredited certification body, since DeployOpen prepares the evidence but does not issue certificates.
How DeployOpen scopes, deploys and supports this evidence
DeployOpen plans and deploys self-hosted Langfuse inside your own infrastructure, then documents how the traces map to the Annex A operation and monitoring and event log controls your auditor will test. Engagements are fixed-scope, with the scope, deliverables and timeline agreed in writing before work starts.
How we scope
- Inventory which AI systems fall within the ISO/IEC 42001 scope and which Annex A controls the statement of applicability includes.
- Define the trace fields, retention periods and masking rules that A.6.2.6 and A.6.2.8 evidence will depend on.
- Agree in writing which applications, environments and deliverables are covered, with a fixed scope and timeline.
How we build
- Deploy Langfuse Web, Langfuse Worker, PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage in your accounts or data centers.
- Instrument the agreed applications with the SDK or OpenTelemetry, and configure alerts for cost, latency and failed calls.
- Test that event records are created, stored and retrieved as documented, including access controls and log retention settings.
How we hand over
- Deliver a control evidence map linking each in-scope Annex A control to the traces, dashboards and runbooks that support it.
- Document operating procedures for monitoring, repairs, updates and support, as A.6.2.6 expects them to be defined.
- Hand over configuration, backup and restore procedures so your own team can run the platform and produce evidence.
How we support
- Offer ongoing support, upgrades and continuous testing as a retainer after handover, if your organization wants it.
- Plan Langfuse version upgrades, including dedicated migration steps for major releases, before they reach production.
- Prepare evidence packs for your internal reviews and external auditors, who alone issue any audit report or certification.
