LLM observability for EU AI Act logging

How LLM observability helps meet EU AI Act logging duties

If you provide or deploy a high-risk AI system in the EU, you need two things from the system itself: automatically generated logs and a post-market monitoring system that collects performance data over its life. Self-hosted tracing can capture the records, but the legal duties stay with your organization. Article 12(1) requires high-risk systems to technically allow automatic recording of events over their lifetime. Article 19(1) requires providers to keep those logs for at least six months, unless other law provides otherwise, and Article 26(6) places a parallel retention duty on deployers for logs under their control. Article 21(2) requires you to give competent authorities access to these logs on reasoned request, so the storage must be somewhere you can actually reach and export from.

Which obligations apply, and when, depends on the system. Under Regulation (EU) 2026/1744, the high-risk rules apply from 2 December 2027 for Annex III systems and from 2 August 2028 for Annex I product systems. Financial institutions may keep logs as part of documentation under Union financial services law (Article 19(2)), and Article 72(4) allows some providers to integrate monitoring into existing systems if protection is equivalent. Confirm your system's category and any sector rules with counsel before you set retention periods. Langfuse can hold traces, scores and metadata in your own PostgreSQL, ClickHouse and object storage, while Phoenix is licensed under the Elastic License 2.0, which prohibits offering it as a hosted or managed service, so check that license limit before you choose it.

Article 72 asks for a documented, proportionate post-market monitoring plan, which the 2026 amendment ties to the Annex IV technical documentation, and Article 9(2)(c) expects risks found in monitoring data to feed your risk management system. Traces, evaluation scores and user feedback can supply that performance data, but the plan, the review cadence, the people who act on findings and the deployer feedback loop under Article 26(5) are decisions you must make and record. Ask your implementation partner how trace retention, role-based access and export are configured, and which Langfuse features need an Enterprise license key, such as data retention policies and audit logs. Note that DeployOpen prepares systems and evidence for assessments but does not issue conformity or audit outcomes.

Which AI Act article requires which logging or monitoring function

This table maps each relevant AI Act article to the logging or monitoring function it asks for, and to the tracing platform settings that support it. Read the last column first when scoping work: it separates retention periods the Act sets from settings you choose and document yourself. The main takeaway is that the Act defines the outcome, while the platform configuration and the retention decisions remain your responsibility.

Article	Function it requires	How to configure the tracing platform	Configuration choice or legal determination
Art. 12(1)	Technically allow automatic recording of events (logs) over the system's lifetime	Define a trace schema that captures each model call, tool call, prompt version, model version and timestamp; instrument at the gateway or SDK so no path skips recording	Design obligation. Event categories and fields are your choice, justified against the intended purpose
Art. 19(1)	Keep automatically generated logs for at least six months, to the extent under your control	Set retention on the trace store and object storage to at least six months for high-risk traces; export to controlled storage with immutability or versioning where available	Minimum period is set by the Act. Anything longer, and deletion rules, are your decision and must be documented
Art. 19(2)	Financial institutions may keep logs as part of documentation under Union financial services law	Map trace exports into the existing financial documentation repository with a stable identifier per record	Legal determination of whether the financial services route applies; confirm with your compliance team
Art. 21(2)	Give competent authorities access to logs on reasoned request	Create a read-only project role for compliance exports; keep export jobs and access records; confirm role-based access and audit logs are available in your Langfuse edition	Procedure is yours to define. Whether a request is reasoned is for the authority to judge
Art. 26(6)	Deployers keep logs under their control for at least six months	Give deployer tenants their own project or storage path; apply the same retention settings and export controls as the provider side	Minimum period is set by the Act. Deployer-side retention settings are configured by the deployer
Art. 72(1) and 72(2)	Run a documented post-market monitoring system that collects and analyzes performance data over the system's life, including from deployers	Tag traces with system version, use case and deployer identifier; schedule evaluation runs and aggregate scores over time; ingest feedback from deployers through an OpenTelemetry Collector pipeline	Obligation to have a system is set by the Act. Metrics, sampling rate and review frequency are your design choices
Art. 72(3)	Base monitoring on a plan that forms part of Annex IV technical documentation; Commission template due by 2 September 2027	Keep the monitoring plan in version control alongside evaluation definitions and dashboards; record each change with a reference to the trace configuration	Plan content is yours to write now. Commission template may change the expected structure; check guidance when published
Art. 72(4)	Integrate monitoring into existing systems where equivalent protection is achieved	Only relevant if an existing monitoring stack already covers these elements; document a mapping of each element to the existing controls	Equivalence is a legal and technical judgment. Confirm applicability with counsel before relying on it
Art. 9(2)(c)	Evaluate risks identified from post-market monitoring data in the risk management system	Route alerts, low evaluation scores and recurring trace failures into the risk register with a ticket reference; link traces to the risk entry	Process design is yours. Frequency of risk review is a documented choice
Art. 26(5)	Deployers inform providers of issues in line with Article 72	Give deployers a feedback channel that attaches trace identifiers and evaluation results to reports; keep the report history in the provider's storage	Reporting thresholds and channels are set in contracts and procedures. Confirm with your legal team

What the six-month retention duty means for trace storage

Article 19(1) sets a floor, not a fixed period. Providers of high-risk systems must keep the automatically generated logs under their control for a period appropriate to the intended purpose, and that period must be at least six months unless Union or national law, in particular data protection law, provides otherwise. Article 26(6) places a parallel minimum on deployers for logs under their control. Article 19(2) allows financial institutions to keep the logs as part of documentation they already hold under Union financial services law. The practical question for your storage design is therefore not whether six months is enough, but what your intended purpose justifies, and whether GDPR storage limitation or sector rules cap how long prompts and outputs containing personal data may stay in traces.

In a self-hosted Langfuse stack, the retention clock is set in two places. Trace and observation data sits in ClickHouse, where retention is handled through the data model and the platform's retention settings, and file and blob payloads sit in S3-compatible storage, where lifecycle rules set expiry for objects. Langfuse data retention policies are not in the open source feature set. Per Langfuse's license key documentation, they require an Enterprise license key, so an organization that needs enforced, per-project retention inside the application must either budget for that license or build deletion jobs and bucket rules itself, and then test that both layers expire data together. Confirm with Langfuse which retention behavior applies to each store in the version you deploy.

Before you set any period, take the question to counsel and your data protection officer. Ask them to record the intended purpose of each logged system, the minimum retention each legal basis supports, whether logs must be kept for a longer period under sector or record-keeping rules, and how erasure requests interact with logs you are required to keep. Write the answer into the logging policy and the post-market monitoring plan so that the storage configuration can be traced back to a decision. Because an automated expiry rule that removes logs too early can breach the Article 19 floor, and one that keeps personal data too long can breach GDPR, test the expiry jobs against the agreed periods before production use.

Which open-source tracing platform fits which case

Langfuse is the option most often considered for recording the automatically generated logs that Article 19 expects, because it can run in the organization's own storage. Its repository is MIT licensed, except the directories named ee/, web/src/ee/ and worker/src/ee/, which fall under a separate enterprise license. Langfuse states that self-hosted deployments use the same codebase as Langfuse Cloud and that core features and APIs are available in the open source version without limits.

License boundary: Core tracing, evaluation and API features are in the MIT part. Confirm with the Langfuse LICENSE file which files your deployment will touch.
Enterprise controls: Project-level RBAC roles, audit logs, server-side data masking, data retention policies and SCIM provisioning need a license key. Confirm each against the current Langfuse license key documentation before you rely on it as compliance evidence.
Components to run: Web and Worker containers, PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible storage, all in your infrastructure.
Production topology: Docker Compose on a single VM is documented for testing without high availability. Production guidance points to Kubernetes.

Before committing, confirm which retention and access controls your Article 19 and Article 21 obligations require and whether they are in the free or licensed tier.

Platforms and components that fit this use case

EU AI Act logging is rarely one product. A trace store, the telemetry path that feeds it, the gateway that sees every model call, and the chat front end that creates user sessions each hold part of the record. The cards below cover the components a self-hosted design usually weighs, and what to confirm about each before you commit.

Self-hosted Langfuse

Langfuse runs as two application containers, Langfuse Web for the UI and APIs and Langfuse Worker for asynchronous event processing, backed by PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage. Together these hold the automatically generated logs you must keep, so the storage sits inside your own accounts. Check which controls you need, such as retention policies and audit logs, because the license key docs list them as Enterprise features.

Arize Phoenix for tracing

Phoenix is an LLM tracing and evaluation tool under the Elastic License 2.0, which prohibits offering the software to third parties as a hosted or managed service. That restriction does not block self-hosting for your own staff, but it matters if you run the system for a customer or a group company. Confirm with counsel how the license applies to your deployment model before you choose Phoenix as the record store.

OpenTelemetry Collector

The OpenTelemetry Collector is Apache-2.0 licensed and receives, processes and exports telemetry from applications. Run it as an agent beside services or as a gateway that collects from many sources, and use it to mask personal data before traces are stored. Authenticator extensions can check OIDC tokens on incoming connections, which helps control who can write records. Check the stability level of each component you use, because the status documentation says components vary.

LiteLLM or APISIX as AI gateway

An AI gateway sits between applications and model providers, so it sees every model call. LiteLLM provides cost tracking and logging across more than 100 LLM APIs under MIT, with its enterprise directory excluded. Apache APISIX adds AI gateway features under Apache-2.0. Use the gateway to write request metadata with a consistent identity, and confirm which fields it logs by default, since full prompt bodies may need to be masked before they are kept.

Open WebUI or LibreChat front ends

Chat front ends create the user sessions that link a person to a prompt, so they hold identity evidence the AI Act record needs. LibreChat is MIT licensed. Open WebUI's license clause 4 restricts removing its branding for deployments above 50 end users in a rolling 30-day period, which matters only if you alter the interface. Confirm what session, user and file events each product writes, and whether you can forward them to your trace store.

Building a post-market monitoring system from trace data

Article 72(1) and (2) require providers of high-risk AI systems to establish and document a proportionate post-market monitoring system that actively and systematically collects and analyzes performance data over the system's lifetime, including data supplied by deployers. Traces are the most direct source of that data for an LLM application: each record can hold the model version, prompt template, retrieval results, output, latency, token counts and cost, and the scores attached by evaluations. The decision to make early is which signals count as performance data for your system, such as error rates, refusals, user feedback, drift in evaluation scores and cost per request, and how each one is written down in the monitoring plan so an assessor can trace it back to a source.

Article 72(3), as amended in 2026, ties this system to a post-market monitoring plan that forms part of the Annex IV technical documentation, and asks the Commission to provide guidance and a template by 2 September 2027. Until that template exists, confirm the plan structure with your legal or compliance team and with the notified body or internal-control process that applies to your system. In practice, sampled traces, scheduled evaluation runs, cost and error dashboards, and a review queue for user feedback should feed a documented loop into the risk management system under Article 9(2)(c), which requires risks identified from monitoring data to be evaluated. Record who reviews each signal, how often, and which change to prompts, models or retrieval triggers a new risk assessment.

Article 72(4) allows providers of certain Annex I products, and financial institutions with Annex III point 5 systems, to integrate these elements into existing monitoring systems if they achieve equivalent protection. If your organization already runs a monitoring stack, such as OpenTelemetry Collector pipelines feeding existing dashboards, confirm with counsel whether that setup qualifies before relying on it. Deployers have their own duty to inform providers under Article 26(5), so agree on the channel for deployer feedback, such as a feedback endpoint or support ticket category, before go-live.

Evidence auditors and authorities ask for from the logging stack

This table lists the evidence items an assessor or market surveillance authority is likely to request for AI Act logging and post-market monitoring, and the component that produces each one. No single item is sufficient on its own; auditors look for the set together, traceable over the system's lifetime, and they should confirm the exact scope with the independent assessor or competent authority for their system.

Evidence item	Producing component	What the assessor checks
Logging configuration baseline	Langfuse Web and Worker settings, OpenTelemetry Collector config files	Which events, fields and trace attributes are recorded, versioned and approved, with the config kept under change control
Retention settings and change history	Langfuse data retention policy (Enterprise license key), infrastructure-as-code history	Retention is at least six months where Article 19 applies, and every change is dated, approved and reversible
Storage and access controls	PostgreSQL, ClickHouse and S3-compatible blob storage configuration	Logs sit in the organization's controlled storage with encryption, backups and restricted administrator access
Role assignments and access reviews	Langfuse project roles (Enterprise license key), SSO and SCIM provisioning records	Only named, authorized people can view traces, and their access is reviewed on a set schedule
Audit logs of administrative actions	Langfuse audit logs (Enterprise license key), Collector and cluster audit trails	Who changed settings, exported data or deleted records, and when, can be reconstructed
Evaluation results	Langfuse evaluation scores, datasets and experiment runs	Scores are reproducible, tied to a model and prompt version, and reviewed by a named owner before a change ships
Post-market monitoring plan	Governance document referenced in the Annex IV technical documentation	The plan describes which performance data is collected, from which sources including deployers, and how often it is analyzed
Incident, drift and feedback records	Trace alerts, evaluation trend reports, deployer and user feedback tickets	Issues feed back into the risk management system and documented corrective actions
Upgrade records for major versions	Langfuse upgrade notes, migration logs, ClickHouse and PostgreSQL version records	Each major upgrade has a tested migration, rollback path and proof that logs stayed complete and readable

What tracing tooling alone does not satisfy

Recording events is one obligation among several. The EU AI Act also asks providers and deployers to classify the system, document how it was assessed, run a monitoring plan and answer authorities, and those duties sit largely outside any trace store. The cards below cover the gaps a self-hosted Langfuse or Phoenix deployment leaves open, and DeployOpen prepares the evidence for each, while independent auditors and certification bodies make any formal determination.

System classification and conformity assessment

Tracing does not decide whether a system is high-risk. That depends on its intended purpose and whether it falls under an Annex III use or an Annex I product. Most Annex III systems use internal-control conformity assessment under Annex VI, while notified bodies are involved for some biometric systems and Annex I products. Confirm the classification in writing before designing the log schema, because it sets which events must be captured and for how long.

Risk management system documentation

Article 9 requires a risk management system that runs across the system's life. Traces show what the model did, but not which risks were identified, accepted or mitigated. Article 9(2)(c) asks the risk process to evaluate risks found through post-market monitoring data, so the team must record how trace findings flow back into the risk register. Ask your partner how evaluation scores and incident reviews are linked to risk entries.

Post-market monitoring plan and governance

Article 72 requires a documented, proportionate monitoring system that actively and systematically collects and analyses performance data. A dashboard is not a plan. The plan should name the metrics, sampling rules, review cadence, owners and escalation paths, and it must cover feedback from deployers. The 2026 amendment places this plan within the Annex IV technical documentation, so the plan needs a version history that auditors can follow.

Deployer obligations under Article 26

Where an organization deploys a high-risk system built by someone else, Article 26 places its own duties on it: using the system as instructed, assigning human oversight, keeping logs under its control for at least six months, and informing the provider of risks. If the deployer runs the tracing stack, its retention settings and access rules must match these duties, and the provider's logging design must be agreed before go-live.

Access for competent authorities

Article 21(2) requires providers to give competent authorities access to logs on a reasoned request, to the extent the logs are under their control. This means the trace store must be queryable, exportable and attributable to a specific system version, with a documented process for who responds and how quickly. Confirm with your partner how Langfuse projects, roles and exports are set up, and which features used for this need an Enterprise license key.

2026 timeline changes

The Digital Omnibus on AI changed the dates. As amended in 2026, the high-risk rules apply from 2 December 2027 for Annex III systems and from 2 August 2028 for Annex I product systems. These dates do not remove the logging duty from design. Systems built now will need their logs, monitoring plan and documentation ready when they enter service, so teams should confirm which deadline applies to their system and plan the work against it.

Questions buyers ask before hiring help for AI Act logging

How much effort does AI Act logging and monitoring take, and what drives it?

The effort depends mostly on how many AI systems are in scope, how many applications and model calls each one makes, how much trace volume they generate, how long logs must be kept, and whether single sign-on and masking of personal data are required. Each of these adds instrumentation, storage sizing, access configuration and testing work. Article 19(1) sets a minimum retention of at least six months for logs under the provider's control, unless other Union or national law, in particular data protection law, provides otherwise, so the retention rule you choose changes the storage plan directly. Ask any partner for a written scope that lists systems, trace sources, retention periods and access roles before agreeing a quote.

What does our team need to provide before implementation starts?

Your team needs to supply decisions that no implementation partner can make for you. Legal or compliance staff should confirm whether each system is high-risk and which regime applies, including whether Article 19(2) lets a financial institution keep logs as part of documentation under Union financial services law. Data owners should set retention periods, decide which personal data may appear in traces, and name who may view them. Engineering owners should identify the applications and model calls to instrument. Without these inputs, the design and the masking rules cannot be finalized.

Can an internal team set up AI Act logging on its own?

An internal team with experience in self-hosted infrastructure and observability can deploy Langfuse or Phoenix alone, and the self-hosted Langfuse documentation covers Docker Compose for testing and Kubernetes with Helm or Terraform for production. The harder parts are usually the regulatory mapping, the retention and masking design, the access model and the evidence file, which need legal, risk and security input. Langfuse's self-hosting docs also list PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage as dependencies that must be run, patched and backed up. Many teams split the work: internal staff own the running platform, and an outside partner handles configuration, testing and documentation.

What changed in the EU AI Act with the 2026 amendment and the 2027 and 2028 dates?

Regulation (EU) 2026/1744, the Digital Omnibus on AI, amended the AI Act on 8 July 2026, was published on 24 July 2026 and entered into force on 27 July 2026. As amended, the high-risk rules in Chapter III Sections 1 to 3 apply from 2 December 2027 for Annex III systems and from 2 August 2028 for Annex I product systems, instead of 2 August 2026 and 2 August 2027. The amendment also changes Article 72(3), which now bases post-market monitoring on a plan that forms part of the Annex IV technical documentation, with Commission guidance and a template due by 2 September 2027. Logging and monitoring design should follow the consolidated text, and your counsel should confirm how these dates apply to each system.

Can we use Arize Phoenix as a hosted service for our customers?

No, not without a separate arrangement with the license holder. Phoenix is licensed under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service. Running Phoenix inside your own infrastructure for your own staff is a different use from offering a tracing service to customers, so the distinction matters if your AI product is delivered to others. Have counsel review the Phoenix LICENSE text against your delivery model before the design is fixed. Langfuse, by contrast, is MIT licensed except for its ee directories, so the license question there is about which features you use.

Which Langfuse features need an Enterprise license key when self-hosted?

The Langfuse license key documentation lists the self-hosted features that need an Enterprise license key. These include project-level RBAC roles, protected prompt labels, data retention policies, audit logs, server-side data masking, UI customization, organization creators, the org management API with SCIM, and the instance management API. Core features and APIs are available in the MIT-licensed version without limits, according to Langfuse. Several of the items above matter for AI Act logging, especially retention, audit logs and access roles, so confirm the current list in the Langfuse license key docs before deciding whether client-side controls are enough. The self-hosted pricing page lists a custom-priced Self-Hosted Enterprise plan. Ask a vendor for its current terms in writing.

What does a fixed-scope AI Act logging engagement cover?

A fixed-scope engagement is agreed in writing before work starts and sets out the scope, deliverables and timeline. For this topic, it can cover planning the logging and monitoring design, deploying a self-hosted tracing platform in your own cloud accounts or data centers, instrumenting agreed applications, configuring retention, masking and access, setting up the post-market monitoring data flows, and preparing the documentation and evidence that your compliance team and auditors will ask for. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications, and independent auditors or certification bodies make those determinations. Ongoing upgrades, support and continuous testing are available as retainers. Ask for the deliverables list and exclusions in the quote.

How does Article 72 post-market monitoring relate to the trace data?

Article 72(1) and (2) require providers to establish and document a proportionate post-market monitoring system that actively and systematically collects and analyzes performance data over the system's lifetime, including data from deployers. Traces, evaluations and user feedback can supply part of that data, but they do not form the whole system. Article 9(2)(c) also requires the risk management system to evaluate risks identified from monitoring data, and Article 26(5) requires deployers to inform providers in line with Article 72. Decide early which monitoring signals come from traces, which come from other sources, and who reviews them. Confirm the mapping with your compliance advisers.

How a logging and monitoring implementation is phased

The sequence moves from legal scoping to a running, hardened trace store and then to evidence that a compliance reviewer or competent authority can read. Each phase closes with a written output, so the record of what was built and why stays with the client.

Scope the system and its legal classification
DeployOpen works with the client's counsel and the system owner to confirm which AI system is in scope, whether it is a high-risk system under the AI Act, and whether the organization acts as provider, deployer or both. The output is a written scope note listing the system boundary, the applicable articles, and the questions counsel must settle, such as whether any other Union or national law sets a different retention period under Article 19(1). Scope, deliverables and timelines are agreed in writing before any build work starts.
Design the trace schema and retention
The team maps which events the system generates automatically, such as prompts, model versions, tool calls, outputs, timestamps and user or session identifiers, and decides what each log must contain to support traceability. Retention is set against the minimum six-month duty in Article 19(1) and Article 26(6), together with data protection limits on personal data. The output is a trace schema document and a retention and access matrix approved by the client's compliance lead.
Deploy and harden the self-hosted stack
DeployOpen installs the chosen observability platform, such as self-hosted Langfuse, inside the client's own cloud account, data center or network, with its database, analytics store, cache and object storage. Hardening covers network isolation, single sign-on, role-based access, encryption, and backup and restore testing for the trace store. The output is a deployment record with configuration, a tested restore, and a list of features that need an enterprise license key, confirmed with the vendor before the design is final.
Instrument applications and wire evaluations
Engineers add tracing to each application or agent that calls the model, using the vendor SDK or OpenTelemetry instrumentation, and apply masking of personal data before records leave the application or collector. Evaluation jobs and scoring rules are configured against the trace data so performance can be tracked over time, which feeds the post-market monitoring duties in Article 72. The output is a test report showing that required events are captured, masked as designed, and visible to the right roles.
Hand over evidence and runbooks
DeployOpen assembles the compliance evidence package: the scope note, trace schema, retention matrix, configuration records, test results, and the monitoring plan the client will maintain. Operational runbooks cover upgrades, incident handling, access reviews, and responding to a reasoned request from a competent authority under Article 21(2). The output is a signed handover pack. DeployOpen prepares this evidence, while independent auditors and certification bodies assess it.

How DeployOpen scopes, delivers and supports AI Act logging

DeployOpen treats AI Act logging as an engineering and documentation task that runs over the system's whole life, not a one-time setup. We scope the logging and monitoring design in writing, deploy self-hosted Langfuse or a comparable tracing stack inside your own infrastructure, and hand over the evidence your compliance team needs for assessment and post-market review.

How we scope

Written scope listing which high-risk AI systems, applications and traces fall under the logging and monitoring design
Mapping of Article 19 retention, Article 21 access and Article 72 monitoring duties to named systems and owners
Fixed-scope engagements agreed in writing before work starts, with pricing set from that scope

How we build

Self-hosted Langfuse deployment with PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage in your own cloud or data center
Instrumentation of applications and an OpenTelemetry Collector path that masks personal data before storage
Retention settings, access roles and test traces that demonstrate automatically generated logs are recorded

How we hand over

Documented architecture, configuration, retention and backup procedures for your compliance and engineering teams
Runbook for monitoring review, evaluation setup and the process that feeds post-market findings into risk management
Evidence pack describing what the logs contain and how they are produced, ready for your auditors to review

How we support

Upgrade planning for Langfuse releases, including dedicated migration guides for major versions and ClickHouse prerequisites
Optional retainer covering ongoing support, upgrades and continuous testing of the logging stack
Clear ownership split: DeployOpen prepares systems and evidence, while independent auditors and authorities assess compliance

LLM observability for EU AI Act logging and monitoring