Dify conversation log retention: what a self-hosted team must decide first
A regulated team running Dify apps should assume that every user prompt and model answer is being kept. Dify keeps logs indefinitely by default, so unless scheduled cleanup is turned on, conversation and workflow records accumulate for as long as the instance runs. Retention has to be set deliberately to match the privacy policy and the records schedule that applies to the business, and that decision belongs before the first user reaches a production app.
The feature covers three kinds of record. Conversation messages are the exchanges between users and a web app or API. Workflow run logs record the execution of chatflow and workflow apps, including the inputs and outputs of each run. Trigger runs, such as those started by webhooks or schedules, produce their own workflow records. Teams need to act when any of these records could contain personal data, customer details or confidential business content, which is common in support, HR and internal knowledge use cases. Operators should confirm with their vendor or in their own environment which records each app type creates, since the cleanup settings do not treat all of them the same way.
Two settings govern this in the self-hosted environment. WORKFLOW_LOG_CLEANUP_ENABLED controls cleanup of workflow run logs, with WORKFLOW_LOG_RETENTION_DAYS setting the period, and ENABLE_CLEAN_MESSAGES controls cleanup of conversation messages, with SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS setting that period. Confirm the defaults for your release in the Dify environment variable reference before writing them into policy. Also note that deleting a conversation does not erase files uploaded in it, so file storage needs its own retention decision.
What Dify records for each web app, API and trigger run
Every Dify app run leaves a trace, but the trace differs by app type and entry point. Knowing which record types exist tells you what to review, what to export, and what cleanup jobs will and will not touch.
Web app and API conversations
Conversation records hold the messages exchanged between end users and a chat app, whether the user arrived through the built-in web app or a service API call. Each record is tied to an app and, for API calls, to the end user identifier the calling product sends. Check which identifiers your integration passes, because they determine whether you can trace one person's history without guessing. Confirm the exact fields stored in your Dify version before writing retention rules that depend on them.
Workflow and chatflow run logs
Workflow and chatflow runs record the execution of each node, including inputs, outputs and intermediate steps for the run. These logs are often the most detailed record of how an answer was produced, and they can contain sensitive content passed through tools or knowledge retrieval. Their cleanup is controlled separately from conversation messages, so you need to configure both. Confirm which node-level values are stored in your version before deciding what must be redacted upstream.
Trigger run records
Runs started by webhook, schedule or integration triggers appear with the workflow runs they create, so an event from another system leaves a traceable record. These runs are useful for incident review, but they can also hold payloads from external systems that carry personal data. Test triggers against non-production data first, and confirm how the trigger source and payload are shown in the log view for your version.
User ratings of answers
Operators can review conversations and see the feedback end users attach to individual answers, such as likes or dislikes. Ratings help identify weak answers, but they are free text or simple signals attached to content that may contain personal data, so they follow the same retention decision as the messages they describe. Confirm whether ratings are removed when messages are cleaned in your version.
Uploaded files stay after deletion
Deleting a conversation does not erase files uploaded during that conversation. A user's document can therefore remain in storage after the chat that contained it is gone, which matters for erasure requests and for retention schedules. Plan a separate cleanup or storage policy for uploads, and verify with a test conversation in your environment which storage location holds the file.
Exports carry configuration only
DSL exports include app configuration, workflow nodes, model parameters, prompts and knowledge base connections, but not usage logs, knowledge base content or third-party API keys. Logs therefore need their own backup and retention plan. Do not assume that a backed-up or migrated app brings its history with it, and confirm the exact export contents against the version you run.
Retention settings and when each one fits
This table compares the two scheduled cleanup switches for workflow run logs and conversation messages, including which records each one removes and how its retention window is set. Read each row against your privacy policy and your incident response needs before turning either switch on. The main takeaway is that shorter windows reduce stored personal data but also shrink the history you can investigate.
| Setting | Workflow run logs | Conversation messages |
|---|---|---|
| Enable switch | WORKFLOW_LOG_CLEANUP_ENABLED set to true turns on cleanup | ENABLE_CLEAN_MESSAGES set to true turns on cleanup |
| Retention window variable | WORKFLOW_LOG_RETENTION_DAYS, default 30 | SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS, default 30 (confirm this name in your release) |
| Data removed | Workflow run records for workflow and chatflow apps once they age past the window | Conversation messages once they age past the window |
| Data not removed | Uploaded files stay in storage, since deleting conversations does not erase them | Uploaded files stay in storage, and the same applies to conversation deletion |
| Default behavior | Logs are kept indefinitely until you enable the switch | Logs are kept indefinitely until you enable the switch |
| Best fit | Teams that must limit how long workflow inputs and outputs persist | Teams that must limit how long user prompts and model answers persist |
| Investigation cost | Shorter windows remove the trail needed to trace a bad workflow run after the fact | Shorter windows remove the transcript needed to reconstruct what a user was told |
| Audit fit | Keep longer where internal audit or incident records require a set period | Keep longer where complaint handling or regulated customer support needs answer history |
| Confirm before enabling | Test cleanup in a non-production workspace and check the logs docs for your version | Confirm which variable controls message cleanup in your release and whether files need separate handling |
How to turn on scheduled cleanup without losing evidence you need
This sequence sets retention periods with the people who own them, enables Dify's scheduled cleanup, proves it works on a non-production instance and protects production data with a backup before the change goes live.
Agree retention periods with privacy and legal owners
Before touching any configuration, the privacy lead and legal counsel should record how long workflow run logs and conversation messages must be kept, and which records must be kept longer because of a hold, an investigation or a regulatory duty. Operators should not choose these numbers alone. The output is a short signed note that lists each log type, its retention period in days and the owner who approved it. Confirm that the note also states what cleanup does not remove, including files uploaded in conversations, which Dify's logs documentation says are not erased when a conversation is deleted, so those files need their own rule.
Set the cleanup variables in the env file
In the .env file used by your Docker Compose deployment, set WORKFLOW_LOG_CLEANUP_ENABLED to true and WORKFLOW_LOG_RETENTION_DAYS to the period approved in the previous step, and set ENABLE_CLEAN_MESSAGES to true with SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS set to the approved message period. The documented defaults for both retention values are 30 days, so an unchanged value means the approved period has not been applied. Keep the file under version control with the change recorded, and do not store the file in a shared location that other teams can edit. Done when the diff shows exactly these keys changed and nothing else.
Restart services and confirm the values loaded
Restart the Compose stack so the api and worker containers read the updated .env file, then check that the running containers received the new values. Use docker compose config or inspect the environment inside the api and worker containers to confirm the keys and numbers match the approved note. Watch the worker logs for startup errors. The check is complete when every cleanup key reads the intended value in both containers and the services report healthy. If a value is wrong, correct the file and restart again rather than editing values inside a running container.
Test cleanup on a non-production instance
Create a non-production Dify instance with the same version and the same cleanup settings, but with a short test retention period that the team agrees is safe for test data. Create test conversations in a web app, run a few workflow runs including one that calls an API and one that uses a trigger, and record their identifiers and timestamps. Wait for the scheduled cleanup to run, then confirm which workflow run logs and messages were removed and which remain. Also confirm that a file uploaded in a test conversation is still present after its conversation is deleted. Record any records that were removed earlier or later than expected and resolve the difference before going further.
Confirm version notes and sample env before change
Before enabling cleanup on production, open the upgrade guide for your target Dify version on the GitHub Releases page and compare your .env file against the current .env.example. Variable names and defaults can change between releases, so a setting that worked in one version may be renamed or ignored in another. Record the Dify version you are running and the version you are moving to, and note any new variables that must be set. The check is complete when the team has a written list of differences and each difference has an owner and a decision.
Back up production before enabling cleanup
Take a full backup of the production metadata database and the file storage before changing any production setting. Cleanup deletes records that cannot be restored by a Dify setting, so the backup is the only way back if the retention period was wrong. Store the backup in a location with the same access controls as the source data, then test that it can be restored into a separate environment. The output is a backup identifier, the restore test result and the time it was taken, written into the change record.
Enable cleanup on production and verify the first run
Apply the tested .env change to production during the agreed change window, restart the services and confirm the values loaded as in the earlier step. After the first scheduled cleanup has run, compare a sample of recent records against the retention note to confirm that records inside the period remain and older ones were removed. Keep the backup and the change record for the period set by the approved retention note, and tell the privacy and legal owners that cleanup is live with the date it started so their records are accurate.
How operators review and rate answers in the logs
Operators find weak answers by working from the conversation record outward. In a self-hosted Dify instance, the logs view lists conversations for each web app and API app, and an operator can open one to read the full message thread. From a message, the operator can move into the workflow run details for chatflow and workflow apps, where each node's inputs, outputs and status show which step produced a poor answer, whether retrieval returned weak context, or whether a tool call failed. Confirm the exact screen names, filters and which run details appear for each app type in your version, because the console layout changes between releases. The Dify docs on logs describe the current views and are the place to start.
End users can rate a response, and those ratings give operators a way to sort the logs toward answers that users disliked. Treat ratings as a triage signal rather than a quality score: a thumbs-down may reflect a correct answer the user did not want, and a missing rating says nothing about accuracy. Pair ratings with the workflow run details before changing prompts or knowledge bases. Confirm whether your version lets you filter by rating and whether ratings appear in exports, since that determines how you select records for review.
Review work sets retention requirements. If a flagged conversation is still being investigated, a scheduled cleanup of messages or workflow runs could remove the evidence before anyone reads it, so the retention window must be longer than your typical review cycle. Set WORKFLOW_LOG_RETENTION_DAYS and SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS with that in mind, and decide who may keep a record past the window and how that exception is documented. Export only the records your privacy policy allows you to keep, and remember that deleting a conversation does not remove files uploaded in it. Check the exact export options and what they include in your version before you rely on them.
Security and privacy risks in stored conversations
A Dify deployment keeps conversation content in several places at once: the metadata database, file storage, the vector store and backups. Deleting a conversation in the console does not necessarily remove every copy, so the risks below need to be checked together when you set retention.
Personal data in prompts
Users often paste names, account numbers, health details or customer records into chat apps without thinking about storage. Dify records the full prompt and model response for each run, so anything a user types becomes part of the log. Before setting retention, ask which data classes users are likely to enter, whether an input filter or moderation step runs before the model, and who is responsible for responding to a data subject request that touches stored messages.
Uploaded files after deletion
Deleting a conversation does not erase files uploaded in it. The conversation record may disappear from the console while the original document stays in file storage. Check which storage provider your instance uses, list where uploads land, and decide whether a separate cleanup job or storage lifecycle rule must remove them on the same schedule as messages.
Workspace member access to logs
Anyone with a workspace role that can open an app's logs can read prompts, answers and ratings from past conversations. Community roles are coarse, so an editor who builds apps may also see production user messages. Review who holds Owner, Admin and Editor roles, decide which people genuinely need log access, and document that decision alongside your retention period.
Metadata database and storage choice
The metadata database holds conversation and workflow run records, while file storage and the vector store hold other derived copies. Dify supports PostgreSQL by default, MySQL and OceanBase for metadata, and several object storage providers for files. Each store has its own deletion behavior and its own backup and replication path, so map every location where conversation content can be written before you decide what cleanup covers.
Backups that keep deleted data
A scheduled cleanup job removes records from the live database, but backups taken earlier still hold them. If backup retention is longer than your log retention period, deleted messages can be restored long after the policy says they are gone. Set backup retention to match the same policy, confirm how backups are expired, and test a restore to check whether cleaned records reappear.
Cleanup settings that match policy
Dify keeps logs indefinitely by default, so nothing is removed until you enable cleanup. Workflow run logs follow WORKFLOW_LOG_CLEANUP_ENABLED with WORKFLOW_LOG_RETENTION_DAYS, and messages follow ENABLE_CLEAN_MESSAGES with SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS. Confirm these values against your privacy policy, record who approved them, and verify after the first scheduled run that old records are gone.
Questions teams ask before setting Dify log retention
How long does Dify keep logs by default?
Dify keeps logs indefinitely by default, so nothing is removed until you turn on a cleanup setting. Workflow run logs are cleaned only when WORKFLOW_LOG_CLEANUP_ENABLED is set to true, and the retention window is controlled by WORKFLOW_LOG_RETENTION_DAYS, which defaults to 30. Conversation messages follow a separate switch, ENABLE_CLEAN_MESSAGES, with a window set by SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS, also defaulting to 30. Check the Dify logs documentation against your installed version, because the default you inherit is the one that applies until you change it. Your privacy policy should set the number, and the configuration should match it.
Can an internal team configure Dify log retention on its own?
Yes, in most self-hosted setups the change is a set of environment variables in the .env file followed by a restart of the services, which an experienced platform or DevOps engineer can make. The harder part is deciding what to keep and proving the change works. Confirm three things before you start: who owns the retention decision in your organization, whether your Dify version exposes the settings you plan to use, and whether you can test cleanup outside production first. If any of those answers is unclear, a partner can help with the design and verification. DeployOpen implements and documents this work inside your infrastructure under a written, fixed scope.
What does setting up scheduled log cleanup cost, and who pays for it?
The cost has three parts: the internal time to decide retention rules, the engineering time to configure and test cleanup, and the ongoing time to review it after upgrades. Setup is usually one-time, while review recurs with each Dify release and each change to your privacy policy. Costs rise when several apps, workspaces or data sources need different retention periods, when legal holds apply, or when you must prove deletion to an auditor. Dify does not publish a price for this work, and the self-hosted Community edition is free, though the Enterprise edition is custom priced, according to the Dify pricing page. Ask any partner for a written quote that lists each of these parts separately.
What does the work need from our team before it starts?
The work needs three inputs from your side. First, retention decisions: the number of days for workflow run logs and for conversation messages, who may extend them, and how exceptions are recorded. Second, a test environment where you can run cleanup with realistic data volumes and confirm what is removed. Third, a verified backup of the PostgreSQL metadata database and file storage, taken before any change, because cleanup deletes records that cannot be restored from the cleanup job itself. Dify's own guidance recommends a backup before every upgrade, and the same discipline applies to deletion jobs. Agree on who approves the change and who checks the results afterward.
How long does setting up log retention take?
The duration depends on how many environments exist, how many apps and workspaces need separate rules, and whether your deployment is a single Docker Compose host or a Kubernetes cluster. A single instance with one retention policy is mostly configuration and testing. Multiple environments, external databases, or Enterprise deployments add review cycles, change approvals and validation steps. No duration is fixed by Dify, so ask for a timeline tied to your written scope and to the environments you name. The Dify Docker Compose documentation describes the baseline install at 2 CPU cores, 4 GB of RAM and Docker Compose 2.24.0 or later, which helps size the test host.
What happens to uploaded files when conversations are deleted?
Deleting a conversation does not erase the files uploaded in it, according to the Dify logs documentation. Message and workflow log cleanup removes the records, but the underlying files remain in the file storage backend, whether that is local storage, S3-compatible storage or another provider listed in the environment variable reference. You need a separate process to find and remove those objects, and that process must respect the same retention rules. Confirm with your storage team how object lifecycle policies and backups treat these files, since backups may keep copies after the live object is deleted.
Does log retention change what auditors can review?
Yes, shorter retention reduces the history available for review, so the retention period should be set with auditors and the business owner of each app. Compliance frameworks ask for evidence that controls operate, and deletion itself can be part of that evidence. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports or certifications, and independent auditors make that determination. Document the retention rule, the cleanup configuration, the test results and each change, then share that record with your auditor early.
Limits and items to confirm in your environment
Dify's log and cleanup behavior depends on settings, record types and release details that differ between versions and deployments. The cards below list the points to confirm before you write a retention policy into configuration, and each one should be checked against the vendor documentation for the version you run.
Record types covered by cleanup
Confirm which record types the scheduled cleanup actually removes. Workflow run logs and conversation messages are handled by separate settings, so a policy that names other records, such as annotations, feedback or moderation results, may not be covered. List every record type your privacy policy names and map each one to a documented setting or to a manual deletion process.
Sandbox-named message variable
The message cleanup setting carries a variable name that refers to the sandbox, even though it governs conversation messages. Confirm in your release's environment reference what this variable controls, whether it applies to production apps, and whether changing it affects records already stored. Test the behavior on a non-production instance before relying on it.
File deletion behavior
Deleting a conversation does not erase files uploaded inside it, so retention settings alone do not remove personal data held in attachments. Confirm whether your version's cleanup touches stored files at all, and plan a separate process for file storage, such as a lifecycle rule on the object storage bucket or a documented manual deletion step.
Backups and copies outside Dify
Scheduled cleanup removes records from the metadata database but does not change backups already taken. Confirm how long your backup snapshots, database replicas and object storage versions keep deleted data, and whether your retention policy must cover them. Check the backup schedule against the retention window so that restored systems do not bring back records you have deleted.
External observability and log exports
Logs sent to an external tracing, telemetry or log platform follow that platform's retention, not Dify's cleanup settings. Confirm which Dify data reaches each external tool, whether prompts and responses are included, and how deletion requests are propagated. Ask the vendor of each tool for its retention controls and document the answers next to the Dify configuration.
Retention defaults in your release
Dify keeps logs indefinitely by default, and the retention windows for workflow runs and messages are configured separately. Confirm the current default values and the variable names in the release you deploy, because minor releases arrive about monthly. Record the values you set, the date you set them and the person who approved them so the configuration matches the policy in audits.
How DeployOpen scopes and delivers Dify log retention
DeployOpen plans Dify conversation logs and log retention as part of a self-hosted deployment inside your own infrastructure, so retention settings match your privacy policy from the start. Engagements are fixed-scope, with the scope, deliverables and timeline agreed in writing before work begins.
How we scope
- Inventory which apps, API consumers and triggers produce logs and messages that your policy covers
- Record the retention periods your privacy and records policies require for workflow logs and messages
- Confirm how uploaded files, which deletion of conversations does not remove, will be handled
- Agree in writing which Dify environments, apps and workspaces the retention work covers
How we build
- Set the workflow log cleanup and message cleanup variables in the environment file for each instance
- Test cleanup on a non-production copy and confirm which records are removed and which remain
- Document how operators review and rate answers, and who can read those logs
- Align file storage and backup retention with the same policy so copies do not outlive the rule
How we hand over
- Provide a retention runbook that names each setting, its value and the owner who approves changes
- Deliver test evidence showing cleanup ran as configured, for your auditors or internal reviewers
- Walk your operators through checking cleanup jobs and handling a request to delete specific records
- Record the configuration in version control so changes to retention go through review
How we support
- Review retention settings when you upgrade Dify, since new releases can change environment variables
- Ongoing support, upgrades and continuous testing are available as a retainer
- Recheck retention after policy changes, new apps or new trigger sources are added
- Quotes follow from the written scope, so ask for the retainer terms alongside the initial scope
