Dify file storage on S3-compatible object storage

Dify file storage on S3-compatible object storage: when your team needs it

A self-hosted Dify that keeps its files on local disk works until you run more than one API or worker node, or until you need retention and backup control over what users uploaded. Dify writes three kinds of data to its file storage: user uploads, knowledge base documents and the keys used for workspace encryption. On a single host, local disk hides the problem. Once the api and worker services run on separate machines, a file written by one node may not be readable by another, and a host rebuild can take documents and keys with it. Object storage moves that data out of the containers into a store your team can replicate, snapshot and expire under its own policies.

File storage is not configured in the web console. It is set through environment variables in the .env file, chiefly STORAGE_TYPE and the provider-specific keys that follow it. The environment variable reference lists four families of options: OpenDAL, S3-compatible storage, Azure Blob and Google Cloud Storage, along with other providers the reference names. Choosing one is a platform decision, not a tuning step, because it determines which credentials Dify holds, which network paths must be open and which backup tool covers the data. Changing the value on a running system does not move existing files, so decide before the first real documents arrive.

Before you pick an option, confirm three things with your own environment and the current reference: which provider your organization already operates and can back up, whether the bucket or container can be reached from every Dify node and from wherever backups run, and who owns the access keys and their rotation. For self-managed storage inside your network, Ceph and MinIO are the two options most teams evaluate, and the choice between them is covered in the sections that follow. Check the Dify environment variables reference against your target version before you write any storage settings.

Which STORAGE_TYPE option fits which environment

This table compares the main file storage choices for Dify by where the bucket or container lives, who operates it, and what you must confirm before setting STORAGE_TYPE. Read each row against your cloud footprint and network boundary, then confirm exact keys and values in the environment reference for your Dify version.

Option	Where the storage lives	Who operates it	What to confirm before you choose
OpenDAL	Backend selected through OpenDAL, which the Dify environment reference lists as a file storage option	Depends on the backend you select, so the operating team is the owner of that service	Which OpenDAL scheme your version supports, the connection settings it requires, and whether your backend is in scope for your assessment
Native S3-compatible storage	AWS S3 or another S3-compatible endpoint that your organization can reach from the Dify api and worker containers	The cloud provider or the team that runs the endpoint, with your team owning buckets, lifecycle rules and access keys	Access key handling, the bucket region, whether the endpoint requires path-style addressing, and that encryption and versioning match your policy
Azure Blob	Azure Storage account and containers in a subscription you control	Azure as the platform, with your team owning accounts, containers, identity and network rules	Which credential method your version supports, the account and container names, and whether private endpoints are required for your network
Google Cloud Storage	Cloud Storage buckets in a Google Cloud project you control	Google Cloud as the platform, with your team owning buckets, service accounts and IAM roles	The service account credential format your version expects, the bucket location, and the IAM role scope granted to the Dify host
Self-run MinIO	An object store inside your data center or private cloud network, reached over a private address	Your platform or infrastructure team, including upgrades, drive layout, backups and TLS certificates	Whether your MinIO release is supported with your Dify version, the endpoint address, path-style addressing, and how credentials are rotated
Self-run Ceph Object Gateway	RGW endpoint on a Ceph cluster inside your own network, exposing S3-compatible APIs	Your storage team, including cluster health, failure domains, RGW users and upgrades	Confirm the Ceph release line and RGW configuration with your storage team, the endpoint and region values, and that path-style addressing works with your Dify version
Air-gapped or offline network	Storage reachable only from inside an isolated network, with no access to public cloud endpoints	Your team, with any S3-compatible or Azure-compatible endpoint hosted inside the isolated boundary	That the chosen option needs no internet access at runtime, that certificates are trusted by the containers, and that backups also stay inside the boundary
Cloud region or residency constraint	A bucket or container in a specific region that must match data residency rules	Your cloud account owner, with the Dify deployment expected to reach that region	The region value your option requires, egress paths from the Dify host, and whether cross-region replication is allowed by your data policy

How to switch Dify to object storage without losing data

This sequence moves Dify file storage from local disk to an S3-compatible bucket in a controlled order, so every existing file is inventoried, backed up and checked before and after the change. The result is a Dify instance that writes uploads and knowledge base documents to your object store and retrieves them successfully.

Inventory local files and settings
The platform engineer with host access records what the current deployment stores on local disk before anything changes. List the storage volume, the uploaded files and knowledge base documents that depend on it, the current STORAGE_TYPE value in the env file, and the workspaces that hold content. Store this inventory with the change record. It is done when someone other than the author can read the list and identify every location that holds Dify files.
Back up the stack before the change
Before editing any variable, the platform engineer takes a full backup of the PostgreSQL database, the local storage volume and the current env file. The Compose upgrade guidance recommends a backup before every upgrade, and a storage change deserves the same discipline because the metadata database records where each file lives. Keep the backup off the Compose host and run a test restore on a separate copy. It is done when the restore test completes and the backup location is recorded.
Confirm how existing files will be carried over
Before switching, the platform engineer checks the Dify documentation for your target release to confirm whether files already on local disk move to the new store or need a separate migration step. Do not assume that changing STORAGE_TYPE copies existing objects. If the docs do not confirm an automatic move, decide whether affected documents will be re-uploaded or migrated with a documented procedure. It is done when the decision is written down and approved by the application owner.
Provision a bucket scoped to Dify
The storage administrator creates a dedicated bucket on the S3-compatible endpoint your team runs, whether that is native S3, MinIO or Ceph Object Gateway inside your network. Create a service identity whose policy allows access only to that bucket, keep public access blocked, and apply the encryption-at-rest setting your policy requires. Store the endpoint, bucket name and access keys in your secrets manager, not in shared notes. It is done when a test write and read succeed with the service identity and fail with any other bucket.
Set STORAGE_TYPE and matching credentials
The platform engineer edits the env file, choosing the STORAGE_TYPE value that matches the provider, such as the S3-compatible option, Azure Blob or Google Cloud Storage, and adds the provider variables the environment variable reference lists for that choice. Confirm the exact names against the reference for your release. Leave SECRET_KEY unchanged, because changing it after deployment logs out all users and affects encrypted OAuth credentials. It is done when a diff of the env file shows only storage-related changes.
Check .env.example for new variables
After any version change, the Compose upgrade guidance says to check .env.example for variables the new release added. The platform engineer compares .env.example for the target release with the live env file and confirms that no new storage variables are required beyond the ones already set. If the release adds one, add it with a value your team has approved. It is done when the comparison is recorded and every new variable has a documented value or a documented reason for its default.
Restart services and read the logs
The platform engineer recreates the Compose containers so the api, worker and web services read the updated env file, then confirms that all services report healthy. Read the api and worker logs for storage connection errors, authentication failures or permission denials on the bucket. Any error here should stop the change and trigger a rollback to the saved env file. It is done when all core services are running and the logs show no storage errors after startup.
Verify upload, document and retrieval
In a test workspace, the application owner uploads a file to a test app, adds a document to a test knowledge base and runs a retrieval query against that document. The storage administrator then lists the bucket and confirms that new objects appear under the Dify-scoped identity and that the local storage volume has not grown. Record the test app name, document name, query and bucket listing. It is done when all three operations succeed and the evidence is filed with the change record.

Protecting keys, buckets and encrypted workspace data

Object storage holds the documents, dataset files and workspace material that Dify depends on, but the bucket is only one part of the protection. Access scope, credential lifecycle, transport encryption and the separate keys that unlock workspace data all need to be planned together, because losing any one of them can break a deployment that otherwise looks healthy.

Scope the bucket policy to Dify

Grant the Dify storage identity access only to the bucket and key prefix that Dify writes to, and deny listing or access to other buckets in the same account or cluster. Shared buckets used by other applications are a common source of accidental exposure. Ask your storage administrator to show the policy in writing, and test that a request outside the prefix is refused before go-live.

Use least-privilege storage credentials

Create a dedicated access key for Dify rather than reusing an administrator or a personal key. Limit it to the object actions the configured storage backend needs, such as reading, writing and deleting objects in the Dify prefix. Record which Dify component uses the key, since the api and worker both touch stored files.

Plan credential rotation

Set a rotation procedure for the storage access key before it is first issued, including who creates the replacement, where it is stored and how the .env values are updated. Restart the api and worker services after changing the values, then confirm an upload and a download succeed. Keep the previous key active only until the new one is verified.

Encrypt traffic to the endpoint

Use HTTPS for the S3-compatible endpoint, including MinIO or Ceph RGW inside your network, so that document content and access signatures do not cross the network in clear text. Confirm that the certificate is trusted by the Dify containers, because a self-signed chain that works on the host may still fail inside them. Check the endpoint setting in your environment reference before go-live.

Back up SECRET_KEY and encryption keys

Back up SECRET_KEY and workspace encryption keys separately from bucket contents. SECRET_KEY signs session cookies and JWTs and encrypts OAuth credentials, and changing it after deployment logs out all users. Workspace keys can be stored locally or in Azure Key Vault. Restoring stored objects without the matching keys leaves encrypted workspace data unreadable, so store key copies in a controlled vault with restricted access.

Test key recovery together

Run a restore drill that rebuilds the bucket contents, the metadata database and the encryption key material together, then confirm that a knowledge base document and a workspace secret can still be read. Ask whether your storage platform supports object versioning or a second copy, and record which recovery point each piece comes from. Repeat the drill after any change to keys or the storage backend.

Running MinIO or Ceph inside your own network

MinIO is the smaller option: an S3-compatible endpoint that a small platform team can run on hardware it controls. Dify uses the S3 interface for uploads and dataset documents, so the decision depends less on feature depth and more on how the endpoint is deployed, patched and recovered. The sources for this page do not describe MinIO's current deployment models, license terms or support tiers, so confirm those with the vendor before you design around them.

Deployment model: Ask the vendor which topologies it supports for production, such as a single node or a multi-node layout with data protection across drives and nodes. A single node has no redundancy of its own.
License and support: Confirm the current license and whether the support terms you need are offered before you commit, because these terms change over time.
Failure domain: Plan backups or replication to a second location. One object store holding every upload is a single point of loss for ingestion and retrieval.
Operations ownership: Assign an owner for TLS certificates, access keys and upgrades, since an outage stops document ingestion for every knowledge base that depends on the store.

Check the deployment model, license and upgrade process in writing before you choose MinIO over the alternatives.

Questions teams ask before moving Dify storage

What drives the cost of moving Dify file storage to object storage?

The cost comes from the work and the infrastructure, and most of it depends on your environment rather than on Dify itself. The main components are the storage capacity you need for uploads, dataset documents and workspace data, the operator time to plan the bucket layout, configure the STORAGE_TYPE settings, and test reads and writes, and the backup and replication design for the bucket. If you run MinIO or Ceph inside your network, add the hardware, network links and the staff who keep that service running. Ongoing costs recur with capacity growth and patching. Ask any partner for a quote that separates one-time migration work from recurring operation, and that names who owns each component.

How long does it take to switch Dify storage to S3-compatible storage?

The duration depends on how much data already sits in local storage, how many storage-dependent features you use, and how quickly your bucket owner can provision access. Teams usually lose time on network path approvals, credential issuance and testing uploads across the features that read files, rather than on the configuration change itself. Dify's environment reference lists the storage options, so the switch is mostly a planning and verification exercise. Ask for a timeline that lists each dependency and the test that closes it, and do not accept a date that skips the migration of existing files.

What does my team need to provide for a storage migration?

You need to provide the people and access that the work cannot proceed without. This typically includes a named bucket owner who approves the bucket layout, retention and access policy, a network path from the Dify containers to the object store endpoint, and service credentials with the minimum permissions for the bucket. You should also supply test data that resembles real uploads and dataset documents, so the checks reflect your content. Confirm with your platform team whether the object store sits behind a proxy or firewall that could block the path. Agree on who receives the credentials and how they are rotated.

Can our own team move Dify storage without a partner?

Yes, a team with Docker Compose or Kubernetes experience, an object storage administrator and a test plan can make the change themselves. The configuration keys are documented, and the work is mostly careful testing. A partner adds value when the team lacks experience with object storage operations, when the storage must meet a compliance evidence request, or when the migration has to run alongside an upgrade. Compare the internal effort, including the time your administrators spend on the object store, with a written fixed scope. Confirm the skills you need before deciding.

Who maintains the object storage service after the migration?

The organization that runs the object store maintains it after the migration, and that party should be named in writing before the work starts. For a cloud provider's native storage, the cloud account owner handles the service and Dify only consumes it. For MinIO or Ceph inside your network, your infrastructure team owns upgrades, capacity, disk replacement, backups and monitoring. Ceph's active releases receive periodic backports and security fixes, so plan a recurring upgrade cycle. Ask who responds to an outage, who approves changes to bucket policy, and who restores data when a bucket is damaged.

How do backups and replication change when Dify uses object storage?

Moving files to object storage does not remove the need for backups, because Dify stores its metadata in a separate database and the files alone do not restore a working system. Plan a backup that covers the database, the object store contents and the configuration, and test a restore together. For replication, decide whether the bucket must exist in a second site, and whether the object store product supports that design. Ceph offers multisite configuration through its Object Gateway, but the design, capacity and failover steps are your responsibility. Ask for a restore test as a deliverable.

How do Dify upgrades affect the storage configuration?

Dify upgrades can change environment variables, so storage settings must be checked after every version change. Dify's Docker Compose docs say to follow the upgrade guide for the target version, to take a backup before each upgrade, and to check .env.example for new variables. Dify ships minor releases roughly monthly, and the 1.17.1 release was dated September 10, 2026. Add a storage smoke test to every upgrade run, confirm that uploads and dataset documents still read and write, and record the configuration you deployed. Ask your partner how they will handle this during each release cycle.

Does the storage choice affect Dify's security and compliance evidence?

Yes, the storage choice changes who can read the data, how access is logged and how encryption is handled. Secrets such as SECRET_KEY remain in Dify's configuration regardless of the storage backend, and changing SECRET_KEY after deployment logs out all users. Object storage access should use a dedicated service account with bucket-scoped permissions, and the bucket should be private with encryption at rest if your platform supports it. Confirm the access logs and key custody with your security team. DeployOpen prepares systems and evidence for assessments, but independent auditors issue any audit report or attestation.

Limits to confirm in your environment before cutover

Several of these limits depend on your chosen storage service, your network and your license position, so none of them can be assumed from the Dify documentation alone. Confirm each item with the storage vendor, your network team or counsel, and record the answer before you change STORAGE_TYPE in a production environment.

S3 operation coverage

Dify reads, writes and deletes objects through its storage layer, so your S3-compatible service must support the object operations that layer calls, including multipart handling for large uploads. Ask the vendor for a written list of supported S3 API operations and test uploads, downloads, deletes and large files against a non-production bucket with the same Dify version you will run. Do not rely on general S3 compatibility claims alone.

Region and endpoint settings

Dify needs the correct endpoint, region and addressing style for your service, and mismatches often appear only after a file is uploaded. Confirm whether your provider expects path-style or virtual-hosted-style requests, whether the endpoint must be an internal name, and whether the region value is required even for private deployments. Record these values in your env file layout so the next upgrade does not reset them.

Migrating existing local files

Files already stored on the local volume do not move to the bucket when you change the storage type. Plan how existing dataset documents and uploaded files will be copied, verify object keys match what Dify expects, and test that a sample knowledge base still retrieves and downloads its documents after the move. Confirm the migration approach with the Dify documentation for your version before you schedule a cutover window.

Lifecycle rules that delete documents

Bucket lifecycle policies, versioning settings and retention holds can expire or remove objects that Dify still references, which breaks knowledge base retrieval and file downloads without an obvious error. Review every lifecycle rule and any automated cleanup on the bucket, exclude Dify prefixes from expiry, and confirm that backups of the bucket are taken on a schedule that matches your recovery targets.

Secrets and access key scope

The access key Dify uses should grant only the bucket and actions it needs, and its rotation must be planned because a changed key stops file access across the platform. Confirm how your provider issues and rotates keys, whether keys can be restricted by bucket and prefix, and where the values are stored. Keep these credentials out of exported app DSL files, which do not carry third-party keys in any case.

Multi-tenant use under the license

The Dify Open Source License prohibits using the source code to operate a multi-tenant environment, where one tenant is one workspace, without written authorization. If you plan to host Dify for several business units, customers or partners in one shared storage design, ask your legal team to review the license text and confirm whether a written authorization or the Enterprise edition is required. Community edition is limited to a single workspace.

How the storage migration works inside a running Dify

Changing STORAGE_TYPE only changes where new writes go. After the change and a restart, new uploads and newly added dataset documents land in the configured store, while files written earlier stay where they were. The old store therefore has to remain readable and reachable until every existing file has been moved deliberately. This makes the cutover a two-part job: configuration for new writes, and a planned copy for history. Before you change anything, confirm in the environment variable reference which provider settings each variable needs, and confirm that the api and worker containers read identical storage settings. A mismatch means one service writes to a location the other cannot see.

Dataset retrieval depends on three things staying consistent: the metadata rows in the Dify database that describe documents and segments, the vector index, and the source files in object storage. A metadata row that points to a file missing from the new store can cause failed re-indexing, broken document previews and failed downloads, and the failure may only appear when a user opens a document. The vector store is configured separately, so a storage change does not move or rebuild it. Plan one set of checkpoints that covers a database backup, a verified copy of the stored files and a retrieval test against a sample knowledge base. Pause dataset uploads during the final copy so that the database and the bucket describe the same state.

The steps, order and rollback for moving storage are covered in the Migrating to Dify group, so they are not repeated here. Decision makers should settle three questions before that work starts: whether historical files must move at all under your retention policy, who owns the copy window, and what the team does if the new bucket becomes unavailable after cutover. Ask for a written plan that names the source and target stores, the checks that gate each stage, and the person who signs off on the switch. Confirm with Dify's documentation which object key structure your version uses, because that structure determines how any copy must be verified.

How to verify that Dify reads and writes the bucket

This sequence proves, in order, that Dify writes uploads to the expected bucket path, reads them back for retrieval, and survives a restart without storage errors. The result is a dated record that the storage configuration works before anyone signs off.

Confirm the storage settings before testing
The platform engineer who owns the Compose or Helm configuration checks that STORAGE_TYPE, the bucket name, region or endpoint, and access keys match the values in the environment file that the api and worker containers actually load. Compare the running containers' environment with the intended file, not just the file on disk. Done when the values match and the record notes which file and container were checked. Confirm the exact variable names for your Dify version in the environment variable reference before editing anything.
Upload a test file through the console
A tester signs in to the Dify console with a non-production account and uploads a small, non-sensitive text or PDF file to a workspace or knowledge base. Use a file you have created for this test so its content is known. Done when the upload completes in the interface without an error message and the file is listed with its name and size.
Find the object under the expected prefix
The storage administrator opens the bucket with the object storage console or an S3-compatible client and looks for the test file under the prefix the team documented for this environment. Check the object size and modification time against the upload. Done when the object is present at the expected path and the team has written down the full object key. If the object lands in a different location, stop and correct the bucket name, prefix or endpoint before continuing.
Create a knowledge base document and run retrieval
A tester creates a knowledge base, adds a document with a known sentence that does not appear elsewhere, and waits until indexing shows as complete in the console. Then the tester runs a retrieval query for that sentence from the knowledge base test interface. Done when the query returns the passage from the uploaded document, and a matching object key exists in the bucket for the dataset file.
Restart the api and worker containers
The platform engineer restarts the api and worker services with the same configuration, without changing any environment values. Then the tester repeats the retrieval query from the previous step. Done when the containers return to a healthy state and the same passage is still retrieved, which shows that Dify reads the stored objects after a restart rather than relying on cached local state.
Check logs for storage errors
The operator reviews the api and worker logs covering the upload, indexing and restart window and searches for storage-related errors such as authentication failures, endpoint or region mismatches, access denied responses and missing objects. Record the time range searched and any matches. Done when the logs show no unresolved storage errors for the test actions. If errors appear, correct the cause and rerun the full sequence from the first step.
Record results and clean up the test data
The tester writes down the Dify version, storage type, bucket and prefix, the object key, the retrieval result and the log review outcome, then names the person who approves sign-off. Remove the test knowledge base and the test objects from the bucket, and confirm the deletion in the bucket listing. Done when the signed record is stored with the deployment documentation and the test data is gone from production storage.

How DeployOpen scopes and delivers Dify storage

DeployOpen plans, implements, documents and supports self-hosted open-source platforms inside your own cloud accounts, data centers or air-gapped networks. For Dify file storage, that means fixing the storage backend, the buckets and the keys before any change touches a running workspace, then handing over the configuration and runbooks your team needs to operate it.

How we scope

Inventory which Dify content lands in object storage: uploads, dataset documents and workspace encryption keys.
Choose the STORAGE_TYPE path with your team, covering OpenDAL, native S3, Azure Blob, Google Cloud Storage or MinIO and Ceph.
Agree the written scope, deliverables and timeline in writing before any work starts, since most engagements are fixed-scope.

How we build

Create buckets, IAM policies and access keys with least privilege, and set the matching environment variables in a tested non-production instance first.
Test uploads, dataset indexing, retrieval and downloads against the chosen backend before cutover, including a failed-write scenario.
Confirm that SECRET_KEY, workspace key handling and SSRF_PROXY settings remain correct after the storage change.

How we hand over

Deliver the configuration, bucket layout, key rotation procedure and network path documentation for your operations team.
Document the backup scope for object storage alongside the PostgreSQL metadata database, with a tested restore procedure.
Walk your engineers through the upgrade check against the Dify release notes and the .env.example file for new variables.

How we support

Provide ongoing support, upgrades and continuous testing through a retainer after the initial delivery is complete.
Re-test storage paths when Dify or the chosen object store releases a version that changes how files are written or read.
Review access, key rotation and bucket policies on a schedule agreed in the retainer, with findings recorded for your audit trail.