Why LLM observability for the public sector is a records and boundary problem
Self-hosted tracing keeps prompt and output records under agency control, but the authorization boundary, retention periods and access rules have to be designed before the first trace is stored. An agency team that must put an AI assistant into service will be asked what the assistant did, for whom, and who could see it. Those answers depend on where the trace store sits, not on which dashboard the team likes. Treating traces as debug data to be cleaned up later is the most common way to end up with records nobody can justify keeping or deleting.
A trace of a government assistant is a record. It can contain citizen names, case details, benefit questions and the model's answers to them, and it documents how an automated tool behaved toward the public. That makes retention, legal hold, access by staff and deletion requests governance questions for the records officer and the privacy office, not only engineering settings. Before choosing a platform, confirm with those owners which trace fields may be stored, how long each category must be kept, who may read full prompts, and how a request to remove one person's data would be carried out across the trace store, object storage and backups.
These requirements narrow the platform choice. Self-hosted Langfuse is the usual candidate because its core features are available under the MIT-licensed open source version and it can run inside the agency's own accounts, but several controls the records team will expect, such as project-level RBAC, data retention policies, audit logs and server-side data masking, need an Enterprise license key, so confirm them early. Arize Phoenix is licensed under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service, so a shared deployment run for other agencies or departments under a managed arrangement needs a license review first.
Which requirements shape the trace store for a government AI assistant
This table maps the public sector requirements that most often reach an LLM trace store to what each one means for where traces are kept, who can read them and what evidence to keep. Read each row against your jurisdiction, data types and authorizing official, because many rows apply only in some agencies or only to some data.
| Requirement | What it means for the trace store | Evidence to confirm or keep |
|---|---|---|
| NIST SP 800-53 Rev. 5 baselines | Low, moderate and high baselines select 149, 287 and 370 controls and enhancements. The baseline sets how many access, audit, encryption and retention controls the tracing stack must inherit. | Selected baseline in the system security plan, mapped control inheritance for Langfuse components |
| FISMA and Inspector General reviews | Inspectors General independently assess agency system security and report to OMB, so trace access logs and configuration history must be reviewable after the fact. | Audit log exports, documented roles, change records for tracing configuration |
| FedRAMP scope under M-24-15 | A cloud-hosted tracing service that stores federal information on behalf of an agency generally falls in scope. A tracing stack run only for one agency on its own cloud tenancy may fall outside the shared-service scope, so confirm with your authorizing official. | Scope determination memo, hosting model, authorization status of any cloud provider used |
| CJIS Security Policy v6.1 | If traces contain criminal justice information, CJI in transit and at rest outside physically secure locations needs FIPS 140-3 validated modules or AES with at least 256-bit keys, and vulnerability scanning at least monthly. | Encryption configuration for ClickHouse, PostgreSQL and object storage, scan reports, SC-13, SC-28 and RA-5 mapping |
| 32 CFR 2002.14(h)(2) and CUI | Where traces hold controlled unclassified information on non-federal systems, agencies set requirements using NIST SP 800-171. The trace store then falls under those safeguards. | CUI category markings, agency-specified safeguards, data flow diagram showing where prompts are stored |
| EU AI Act Articles 12(1) and 19(1) | Only where the agency operates in the EU and the system is high risk. Logs must be recorded automatically over the system's lifetime and kept for at least six months unless other law sets longer periods. | Retention settings in the trace store, log export procedure, legal review of the retention period |
| ADA Title II and WCAG 2.1 AA | Applies to user-facing assistant interfaces of state and local governments. Trace review dashboards used by staff are generally not public web content, but confirm this with counsel. | Accessibility conformance report for the assistant UI, applicability assessment for internal tools |
What a trace store holds that auditors will ask about
A trace store for a government AI assistant holds far more than model telemetry. It records what citizens or employees typed, the documents a retrieval step pulled in, the model's answers, user and session identifiers, token counts and cost, and any evaluation scores attached to a response. That combination makes the trace store itself a system of record that must be classified, encrypted, retained and access-controlled like the service it observes. Auditors will ask where each of those fields lives, who can read them, how long they are kept and what happens when a retention period ends. Before design starts, the agency should inventory which fields are personal, which may carry case or benefits data, and which are safe to share with engineering teams.
Retention and access rules should be set per project, not once for the whole Langfuse instance. A benefits chatbot, an internal policy assistant and a fraud-related pilot will have different data, different owners and different legal bases, so each needs its own retention window, its own role assignments and its own review of who can open raw traces. In self-hosted Langfuse, project-level RBAC roles, data retention policies and audit logs are Enterprise features that require an Enterprise license key, according to the Langfuse license key documentation. The open source tier covers the core tracing features, so the agency must decide whether it needs those controls, budget for the license, and verify the key is active in a test environment before any design is finalized.
Choosing a hosting pattern for agency traces
In this pattern the agency owns the hardware, the facility and the operating environment, so there is no cloud service provider for the trace store. Langfuse runs on agency servers alongside PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage, and the agency's existing controls for facilities, hardware and personnel cover those components. Because FedRAMP's scope is cloud products and services that create, collect, process, store or maintain federal information on behalf of an agency, a trace store the agency runs itself does not fall into that scope under M-24-15. The authorizing official will still expect the trace system to be documented inside the authorization boundary of the AI service it observes, with its controls mapped to NIST SP 800-53 Rev. 5.
- No cloud provider in the chain: The agency is the only operator, so the evidence trail runs to its own change, access and facility records.
- Boundary placement: Decide whether the trace store belongs to the same boundary as the AI assistant or to a separate system with a documented interconnection.
- Storage sizing: Plan ClickHouse disk and memory, and the object store for blobs, as owned capacity, since no provider scales them for you.
- Operations ownership: Patching, backups and Langfuse upgrades become agency tasks, including the v3 to v4 migration guide.
- Hardware and key custody: Confirm who holds encryption keys and whether the storage meets the FIPS 140-3 requirements your data category imposes.
Confirm with your authorizing official which boundary the trace store sits in before any build starts.
Here the agency or its integrator operates the trace stack in a cloud environment dedicated to that agency, so the cloud service provider is the infrastructure vendor under its own authorization. Under M-24-15, a cloud service used only for a single agency's operations and not offered as a shared service is out of FedRAMP scope, so the agency's own authorization process governs it rather than a FedRAMP package. If the environment is offered to other customers as a shared service, or the provider's offering as a whole handles federal information for an agency, FedRAMP scope applies and the provider's authorization status and package need to be checked. The authorizing official will want the provider's documentation for inherited controls, the agency's responsibility matrix, and evidence that the trace data stays inside the boundary.
- Shared versus single-agency: Confirm in writing whether the environment is a shared service, since that single fact can bring FedRAMP into scope.
- Inherited controls: Map which NIST SP 800-53 controls the cloud provider covers and which the agency or integrator must still implement.
- Provider authorization: Ask for the provider's FedRAMP authorization details, if any, and the impact level those cover.
- Region and access: Confirm where trace data, backups and object storage are located and which personnel can reach them.
- Integrator role: Clarify whether the integrator operates the stack under the agency's authority and how that is written into the contract.
Confirm the FedRAMP scope question with the agency's authorizing official and procurement office before selecting the provider.
A fully disconnected network has no path to any external cloud service, so no cloud service provider is involved in running the trace store. The stack, its container images, its Helm charts or Compose files, and its model dependencies all cross into the enclave through a controlled import process. FedRAMP scope does not arise for the trace store because nothing is hosted by an outside provider, but the authorizing official will look closely at how software enters the enclave, how updates are validated, and whether any evaluation models run locally. Air-gapped operation also means every Langfuse release must be imported, which makes upgrade discipline a compliance matter, not only an operations task.
- No external provider: The evidence centers on the agency's own accreditation, transfer procedures and media handling.
- Import chain: Record how images, charts and dependency packages are verified before they cross the gap.
- Upgrade cadence: Plan for importing each Langfuse release, and confirm how migrations and rollback are tested offline.
- Local evaluation models: Any LLM-based scoring must run inside the enclave, so its model artifacts go through the same import chain.
- Log and monitoring feeds: Confirm how audit records leave the enclave for the agency's central monitoring, if they do at all.
Confirm the import and update procedure with your accrediting official before the first deployment, because it shapes the whole evidence set.
Langfuse documents a Docker Compose deployment on a single virtual machine, and that option is meant for testing. It runs without high availability, so a single host failure stops trace ingestion and can lose data that has not yet been persisted. It can help an agency prove that an integration works and that traces contain the fields it expects, but it is not a production pattern for records that may need retention and access controls. Whichever hosting model is chosen, the production design needs a documented high-availability approach, which Langfuse describes through its Kubernetes and Helm or Terraform-based deployments on AWS, Azure and GCP.
- No high availability: One VM means one point of failure for Langfuse Web, Langfuse Worker, PostgreSQL, ClickHouse, Redis or Valkey, and blob storage access.
- Test data only: Do not load production prompts or outputs that contain personal or sensitive data into a Compose test instance.
- Promotion path: Plan the move from test to production before testing starts, including data migration and configuration.
- Resource floors: Langfuse's scaling guide lists minimum CPU and memory for each component, and the ClickHouse figures apply to production sizing as well.
- Documentation: Label the Compose environment as a test system in the boundary documentation so it is not mistaken for an authorized production service.
Confirm the production target and its high-availability design with your security team before any agency data flows into a trace store.
Which open-source platforms fit a government tracing stack
A government tracing stack usually combines a trace store, a collector that receives and cleans telemetry before storage, and a gateway that emits consistent logs from every model call. The licenses on each component decide who may run it, for which users and under what support model, so confirm those terms before the architecture is fixed.
Self-hosted Langfuse
Langfuse is the trace store and evaluation layer most likely to sit at the center of this stack. Its core features are MIT licensed and run in the agency's own environment from the same codebase as Langfuse Cloud. Features such as project-level RBAC, audit logs, data retention policies and server-side data masking need an Enterprise license key, so audit access rules and retention before you design around them. Confirm the license key terms with Langfuse before the scope is signed.
Enterprise directories under separate terms
The Langfuse repository carries a separate enterprise license for its ee directories, so the code an agency deploys is not uniformly MIT. Procurement and legal teams should map which components they run, which directories they touch and whether any enterprise feature is needed for access control or retention. Ask the implementation partner to produce a written list of licensed components for the bill of materials, and to confirm the terms before any enterprise feature is enabled in production.
Arize Phoenix for agency-internal use
Phoenix is an LLM tracing and evaluation tool under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service. An agency running Phoenix inside its own boundary for its own staff is the case to confirm, not assume. If a contractor operates the tracing service for several agencies, that arrangement may fall under the license restriction. Confirm the permitted use with counsel and the license text before choosing Phoenix over Langfuse.
OpenTelemetry Collector for masking
The OpenTelemetry Collector is Apache-2.0 licensed and receives, processes and exports telemetry without tying the application to one backend. It is the natural place to filter or mask prompt content before traces reach storage, which matters when prompts carry citizen records. It can also authenticate incoming connections, for example with OIDC. Confirm which processors and extensions your chosen distribution includes and how their stability is documented, since component maturity varies.
LiteLLM or APISIX as the log source
A gateway between applications and model providers is the single point where each model call can be logged in a consistent format. LiteLLM provides cost tracking, guardrails and logging across more than 100 model APIs, and Apache APISIX is an Apache-2.0 API and AI gateway that fits agencies already running it. Verify which logging, redaction and audit fields each gateway emits, and whether the gateway itself is inside the authorization boundary. Confirm the enterprise directory terms for LiteLLM before use.
How an agency rollout moves from authorization boundary to live traces
This sequence takes an agency from a defined authorization boundary to a traced government assistant, with each phase producing a record the authorizing official and assessors can review. The order matters because hosting, identity and retention decisions constrain every later step.
Define scope and classify the trace data
The agency and DeployOpen identify which assistants, services and users are in scope, and classify the prompts, retrieved documents and outputs the traces will hold. Classification shows whether any content is controlled unclassified information, criminal justice information or personal data, because each category brings different safeguards. Ask the agency's information security office and its records officer to confirm these categories in writing. The step closes with a signed scope statement and a data inventory that lists each trace field and its sensitivity.
Decide the boundary and hosting model
The agency decides where the Langfuse stack will run, whether in an agency data center, a government cloud account it controls, or a cloud environment with an existing authorization. DeployOpen documents which components sit inside the boundary, including the web and worker containers, PostgreSQL, ClickHouse, Redis or Valkey, and S3-compatible blob storage. Confirm with the agency's authorizing official whether any hosting provider must already hold an authorization, since FedRAMP scope depends on whether the service is offered to multiple agencies. The step closes with a boundary diagram and a written hosting decision.
Size the architecture from documented minimums
DeployOpen sizes the deployment against the minimums in the Langfuse scaling guide, which lists 2 CPU and 4 GiB for web, worker and PostgreSQL, 1 CPU and 1.5 GiB for Redis or Valkey, and 2 CPU and 8 GiB for ClickHouse, with at least 16 GiB of ClickHouse memory for larger deployments. Expected trace volume, retention length and the number of agents or tool calls per request then drive the storage and memory numbers. Single-VM Compose is suitable for testing only, so production designs use the Kubernetes paths the documentation describes. The step closes with a sizing workbook that the agency can challenge before procurement.
Design identity, SSO and project roles
The team maps agency identity sources to Langfuse sign-in and defines which staff may view traces, prompts and evaluation scores by project. Project-level RBAC roles, SCIM provisioning and audit logs are enterprise-tier features in self-hosted Langfuse, so the agency must decide early whether it needs an Enterprise license key and confirm the terms with the vendor. Developers, reviewers and auditors usually need different access, and the design records who approves access changes. The step closes with an access matrix and a tested single sign-on flow.
Set masking and retention policy before ingestion
The agency's privacy office and records officer set rules for what is masked before data reaches storage, how long traces are kept, and how erasure requests are handled. Masking can run in the instrumented application or in the OpenTelemetry Collector, which moves the control out of the vendor product. Server-side masking and retention policies are enterprise features in self-hosted Langfuse, so the chosen design determines which controls depend on the license. Retention must also satisfy any applicable records schedule. The step closes with a signed policy and masking tests run against sample prompts that contain synthetic personal identifiers.
Instrument the assistant over OpenTelemetry
DeployOpen instruments the assistant and its model gateway so each request produces traces that carry model, latency, token and cost fields. Using OpenTelemetry keeps the telemetry format vendor-neutral and lets the Collector receive, process and export data in one controlled path. The Collector can authenticate incoming connections, for example with OIDC, and add credentials to outgoing requests, and its configuration should follow the published security practices. The step closes with a test run in a non-production environment where the team confirms that no field outside the approved list is captured.
Map controls and collect evidence
DeployOpen maps the configured controls to the relevant NIST SP 800-53 Rev. 5 control families and the baseline the agency has selected, using the control IDs and titles rather than quoted standard text. For each mapped control, the team records configuration exports, access reviews, masking test results, backup records and log samples. The agency's assessors decide whether the evidence is sufficient, because DeployOpen prepares evidence but does not issue assessments. The step closes with a control matrix in which every row points to a stored artifact.
Support the authorization package
DeployOpen assembles the system description, architecture diagrams, data flow records, control implementation statements and evidence index for the agency's authorization package. The agency's authorizing official and independent assessors review the package and decide on any authorization. DeployOpen answers technical questions and updates documents when findings require changes. The step closes when the agency confirms the package is complete for its review and the open items are recorded with owners.
Run controlled upgrades with prerequisite checks
After go-live, upgrades follow a fixed process. Updates within a major version apply database migrations at startup, but major upgrades, such as v3 to v4, follow dedicated guides. The v3 to v4 guide requires ClickHouse 25.12 or later (26.4 recommended), PostgreSQL 15 or later and Redis 7.0 or later, and advises about 3x ClickHouse disk headroom when backfilling historic data. The team verifies these prerequisites, tests the upgrade in a staging copy inside the boundary, and records the change for the authorizing official. The step closes with a change record and a tested rollback procedure.
Evidence assessors and authorizing officials ask for from trace systems
This table maps each evidence item an assessor or authorizing official is likely to request to the component that produces it. Read it as a collection checklist: confirm each artifact exists, who owns it, and where it is stored inside the authorization boundary. DeployOpen prepares these artifacts, while independent assessors and the authorizing official decide whether they meet the agency's requirements.
| Evidence item | Produced by | What the assessor checks | Confirm before assessment |
|---|---|---|---|
| Access logs | Langfuse audit logs, SSO provider logs, Collector authentication logs | Who viewed or changed traces, prompts and projects, and when | Audit log export is configured, and confirm which tier includes audit logs |
| Role configuration | Langfuse project-level RBAC roles and exported role assignments | Least-privilege access to traces containing citizen or case data | Confirm which RBAC roles need an Enterprise license key |
| Retention settings | Langfuse data retention configuration, ClickHouse TTL settings, blob lifecycle rules | Retention matches the agency records schedule and any AI logging duty | Confirm the retention period with the records officer, not the platform default |
| Encryption configuration | ClickHouse, PostgreSQL and S3-compatible storage encryption settings, TLS on Collector receivers and exporters | Encryption at rest and in transit, with FIPS-validated modules where CJI or similar data applies | Confirm module validation status with the vendor and the applicable baseline |
| Software inventory and signed artifacts | SBOM files, SHA256 checksums and signatures shipped with OpenTelemetry Collector releases | Provenance of deployed images and binaries, and a current inventory | Verify checksums and signatures before deployment and record the versions in use |
| Vulnerability scanning results | Container and host scanners run against Langfuse, Collector and database images | Scans at the frequency the baseline requires and documented remediation | Confirm scan cadence and ownership against the applicable control baseline |
| Change records for upgrades | Upgrade tickets, migration logs, pre- and post-upgrade test results | Changes were approved, tested and reversible, including schema migrations | Major upgrades follow dedicated migration guides, so plan them as separate changes |
| SIEM log export | Collector pipelines forwarding audit and access events to the agency SIEM | Events arrive in the SIEM with consistent timestamps and retention | Confirm the SIEM ingestion path stays inside the authorization boundary |
What agencies ask before hiring help to trace LLM assistants
What drives the cost of self-hosted LLM tracing for a government agency?
Cost is driven by four components: infrastructure, the enterprise license decision, the number of applications to instrument, and the boundary assessment work. Self-hosted Langfuse is MIT licensed for its core features, but project-level RBAC, audit logs, data retention policies and server-side masking need an Enterprise license key, and Langfuse prices that plan on a custom basis. Infrastructure covers PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible storage, and Langfuse's scaling guide lists minimum resources for each. Ask any partner for a quote that separates one-time work from recurring costs, names who pays for each license, and states what the boundary evidence work includes.
How long does a government LLM tracing deployment take?
The timeline depends mainly on the authorization boundary decision, more than on installing the software. A deployment inside an existing, already authorized environment moves faster than one that needs a new boundary, new components or a cloud service assessed for federal information. Under FedRAMP's scope (OMB M-24-15), a cloud product that stores federal information on behalf of an agency is in scope, while a system used only for one agency's operations on cloud infrastructure and not offered as a shared service is out of scope. Ask for a written scope with a timeline that starts after the boundary and data classification are confirmed, and ask who in the agency approves each stage.
What must the agency provide before a partner starts work?
The agency must supply a data classification for the prompts, outputs and files that traces will contain, an identity provider to manage access, and a named contact for the authorizing official. Classification matters because the FBI CJIS Security Policy sets requirements for criminal justice information, and version 6.1 took effect on June 26, 2026. Identity matters because access to traces must follow role rules, and Langfuse's SSO and project roles depend on the setup chosen. The authorizing official decides whether the boundary, the logging design and the retention period are acceptable. Ask the partner to list these inputs in the written scope before work begins.
Can agency staff run self-hosted LLM tracing after the engagement?
Agency staff can run it themselves if they have platform engineering skills and a clear runbook, but most agencies keep a support arrangement for upgrades and incidents. Langfuse says updates within a major version apply database migrations automatically at startup, while major upgrades follow dedicated migration guides, such as the v3 to v4 guide that requires ClickHouse 25.12 or later. A retainer typically covers upgrades, monitoring, patching, backup checks and documentation updates, which keeps the evidence current. Ask the partner which tasks are handed over, which stay under the retainer, what response expectations are written down, and whether the retainer covers boundary re-assessment after changes.
Can LLM traces stay out of a US SaaS tracing service?
Yes, traces can stay out of a US SaaS service when the tracing platform runs on infrastructure the agency or its partner controls. Self-hosted Langfuse runs the same codebase as Langfuse Cloud, so the storage location is set by where the agency deploys PostgreSQL, ClickHouse and object storage. Phoenix is licensed under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service, so the agency should confirm that its own deployment fits the license. Ask the partner to show where every trace component runs, where backups are kept, and which vendor support channels could receive any data.
How do OpenTelemetry and the Collector limit what reaches trace storage?
The OpenTelemetry Collector can receive, process and export telemetry from many sources, which gives the agency a single control point before data reaches storage. Its processors can remove or change attributes, and its authenticator extensions can authenticate incoming connections, for example with OIDC. Engineers should confirm which masking rules run in the SDK and which run in the Collector, because enterprise server-side masking in Langfuse is a licensed feature. Ask the partner to show the Collector configuration, the masking test results and the exporter credentials handling, and to confirm which Collector components are stable enough for production use.
Which federal control baseline should traces be assessed against?
Traces should be assessed against the control baseline that matches the impact level of the system they support, using NIST SP 800-53 Rev. 5 with SP 800-53B. The baselines select 149 controls and enhancements at low, 287 at moderate and 370 at high. Audit and accountability controls and access control controls will govern trace logs, and the agency's authorizing official sets the final baseline. Ask the partner to map each trace control to a control ID in the agreed baseline, to state which evidence it produces, and to identify which gaps the agency must close itself.
Questions to put to an implementation partner before signing
Start with the boundary. Ask the partner whether they will operate the Langfuse stack inside the agency's authorization boundary or in their own environment, and if the latter, how the trace data path crosses into the agency's boundary and who is responsible for each segment. Ask which NIST SP 800-53 baseline they plan against (low, moderate or high, as defined in SP 800-53B) and for the control mapping they will produce for the controls they inherit from the hosting platform, the agency and the application. Confirm they can show that mapping in OSCAL or a comparable machine-readable format, and that the agency's authorizing official accepts the baseline before build starts.
Next, ask who holds the Langfuse enterprise license key and who pays for it. Several controls that public sector reviewers look for, including project-level RBAC, data retention policies, audit logs and server-side data masking, need a license key on self-hosted Langfuse, so the answer changes the budget and the design. Ask how masking is applied: in the application SDK, in the OpenTelemetry Collector using processors the partner has tested, or both. Ask for the list of fields masked, the test evidence that masking holds on sample traces, and how the partner handles prompts that include citizen identifiers.
Finally, ask how ClickHouse and object storage are backed up, where the keys are held, and which encryption standard applies. Ask how the partner tests Langfuse upgrades, including the v3 to v4 migration, in a non-production copy before an authorized system changes, and who approves the change record. Request the evidence package in advance: what it contains, which artifacts are generated automatically, and who signs it. Confirm that the partner prepares evidence for the agency's assessor but does not issue the assessment or the authorization decision, which remain with the agency and its independent assessors.
Keeping traces accountable after go-live
After go-live, the operating cycle keeps access, retention, scanning, upgrades and evidence aligned with the agency's authorization boundary, so the trace store can be shown to be controlled on the day an assessor asks. Each step produces a record that can be reviewed, not just a configuration that was set once.
Review access against RBAC roles
The agency's platform owner and security team review who holds each project role in the self-hosted Langfuse instance, comparing current accounts against the approved list for each application and environment. Project-level RBAC roles and the related audit logs are Enterprise license features, so the agency should confirm early whether it needs that license before depending on them. Accounts that no longer map to an approved role are removed, and the review record is signed and filed. The step is complete when the reviewed list matches the access approval records and every change has a ticket reference.
Enforce retention and record deletions
The records owner applies the retention period set in the agency's records schedule to prompt and output traces, scores and attached datasets in ClickHouse and the S3-compatible blob store. Where the instance has data retention policies enabled through the Enterprise license, the policy handles expiry; otherwise, the team runs and logs a documented deletion procedure. Each deletion run produces a record showing the scope, the date range, the operator and the approval. The step is complete when a sample of deleted traces cannot be retrieved from the UI, the API or the backups within the documented window.
Scan and patch at the baseline cadence
The security team runs vulnerability scans on the Langfuse containers, PostgreSQL, Redis or Valkey, ClickHouse and the host operating systems at the frequency the agency's authorization baseline requires, and tracks the findings to remediation. For agencies working under the FBI CJIS Security Policy, version 6.1 calls for vulnerability scanning at least monthly, so the scan schedule should be confirmed against the applicable policy version. Findings are triaged by severity, and the outcome of each finding is recorded. The step is complete when every open finding has an owner, a due date set by the agency and a documented disposition.
Plan upgrade windows around release type
The platform team schedules upgrades in two categories. Updates within a major version are described by Langfuse as non-disruptive, with database migrations applied automatically at startup, so they can follow the normal change process after a test run. Major upgrades, such as the v3 to v4 move, follow the dedicated migration guide, which requires ClickHouse 25.12 or later, PostgreSQL 15 or later, Redis 7.0 or later and extra ClickHouse disk headroom for backfills. Each change record names the target version, the rollback plan and the test evidence. The step is complete when the change is approved, tested in a non-production copy and logged in the configuration baseline.
Confirm log integrity and availability
The operations team checks that traces are still arriving from every instrumented application, that the collector and storage components are healthy, and that backups of the trace store restore to a test environment. Gaps in trace ingestion are investigated because they can mean missing audit evidence for the period. The restore test and the ingestion report are stored with the monitoring evidence. The step is complete when the team can show a continuous record of ingestion for the reporting period and a successful restore test.
Refresh evidence for reauthorization
Before each reauthorization or assessment, the compliance lead assembles current evidence for the controls that the trace platform supports, including access reviews, retention records, scan results, change records and the architecture diagram showing the authorization boundary. Evidence is checked against the current control baseline in NIST SP 800-53 Rev. 5 and the agency's own system security plan. The compliance lead confirms that each piece of evidence is dated, traceable to its source and within its review period. The step is complete when the evidence package has been reviewed against the system security plan and any gaps are logged as tracked remediation items.
Close the loop with the authorizing official
The system owner reports the results of the cycle to the authorizing official and the security governance body, covering open findings, deleted data, upgrades completed and any changes to the authorization boundary. Decisions about risk acceptance are recorded, and any change that affects the boundary triggers a review before it goes live. The step is complete when the report is accepted and the decisions are filed in the system's governance record, which feeds the next access review and scan cycle.
Integration points that touch the agency's existing controls
A self-hosted trace store only satisfies public sector controls when it connects cleanly to the systems that already enforce them. The six points below cover identity, ingestion, security monitoring, routing, retrieval and recovery, and each one should be confirmed with the agency's security and records teams before design is final.
Agency identity provider for Langfuse
Langfuse sign-in and access roles should follow the agency's directory, so that staff leaving a program lose trace access in the same cycle as other systems. Self-hosted Langfuse documents SSO options, but several governance features, including project-level RBAC roles and SCIM provisioning, need an Enterprise license key. Confirm which of those the agency requires before committing to an approach, and record who approves trace access for each program.
OIDC authenticators on Collector receivers
When applications send OpenTelemetry data to a Collector, the receiver should authenticate each sender rather than accept any connection on the network. The Collector supports authenticator extensions that verify incoming connections, for example with OIDC, and add credentials to outgoing exporter requests. Check which service identities are issued for each application, and test that a sender without a valid token is rejected before go-live.
SIEM export of access and security events
Logins to the trace platform, role changes, export actions and deletion requests are security-relevant events that the agency's monitoring should see. Forward them from Langfuse audit logs and infrastructure logs into the agency's SIEM rather than leaving them on the trace host. Agree the event list, the retention in the SIEM and who reviews alerts, since the trace platform itself is not the place where those alerts are handled.
AI gateway for routing and budgets
An AI gateway between assistants and model providers is where the agency can enforce which models are called, which budgets apply and which requests are logged. If the gateway sends its own traces, the agency gets one consistent record of model use, not two partial ones. Confirm that the gateway writes the same request identifiers that the application traces use, so an incident can be followed from the user request to the model call.
Document store feeding retrieval
Retrieval-augmented assistants pull passages from agency document stores, and the traces then contain those passages. The source repository's access rules do not automatically carry into the trace store, so a user who cannot open a document should not be able to read a trace that quotes it. Map each document collection to the trace projects that may see it, and check this mapping during access reviews.
Backup target inside the boundary
Langfuse depends on PostgreSQL for metadata, ClickHouse for trace data, Redis or Valkey for queues, and S3-compatible storage for blobs. Each of these needs a backup target that sits inside the authorization boundary and carries the same encryption and retention rules as the primary data. Ask for the restore procedure to be tested with all stores together, because a ClickHouse backup without matching blob references does not give a usable trace record.
How DeployOpen scopes and supports public sector tracing
DeployOpen plans, implements, hardens, backs up, documents and supports self-hosted Langfuse inside the agency's own infrastructure, whether that is a cloud account the agency controls, a data center or an air-gapped network. In practice that means the work covers the deployment of the Langfuse Web and Langfuse Worker containers, PostgreSQL, ClickHouse, Redis or Valkey and S3-compatible blob storage, the instrumentation of the assistant or service, the access model, backup and restore testing, and the runbooks an operations team needs after handover. DeployOpen also prepares the system, configuration and evidence that an assessor will ask for, such as the description of where traces are stored, who can view them and how long they are kept. DeployOpen does not issue audit reports, FedRAMP authorizations, ATOs or any other attestation. Those decisions belong to the agency's authorizing official, its assessors and the relevant certification bodies, and the scope should say so in writing.
Most engagements are fixed-scope. Before work starts, the agency and DeployOpen agree in writing on the systems in scope, the deliverables, the environments, the evidence to be produced and the timeline. That written scope is also the basis for a quote, because DeployOpen does not publish a price list. Agencies should ask for the quote to state which Langfuse features are assumed, in particular whether the work needs an Enterprise license key for project-level RBAC, data retention policies, audit logs or server-side masking, since the license is a separate purchase from the implementation. Ongoing work such as upgrades, security patching, backup verification and continuous testing can be covered by a retainer, which is usually a better fit for traces that must stay available for years than a one-time project. Agencies should confirm what the retainer includes, how quickly changes are handled and who in their own organization approves each change.
Before signing, the agency should check that the scope names its own boundary owner, the system security plan or equivalent document it will feed, and the independent assessor who will review the evidence. The agency should also confirm whether DeployOpen's support and testing work will itself be assessed as part of the boundary, since personnel and tools that touch trace data may fall within the assessment. A useful test of the written scope is whether an assessor could trace every stored prompt, output and cost record back to a named component, a named access role and a named retention rule.
Sources and further reading
- NIST SP 800-53 Rev. 5 (CSRC) ↗
- NIST SP 800-53B, Control Baselines (CSRC) ↗
- OMB M-24-15, Modernizing the Federal Risk and Authorization Management Program ↗
- FedRAMP Consolidated Rules for 2026: Scope of FedRAMP (M-24-15) ↗
- California DOJ Information Security bulletin 2026-ISRS-001: FBI CJIS Security Policy version 6.1 ↗
