Running several Dify instances on shared infrastructure: when it makes sense
A platform team that wants development, staging and production Dify without three full stacks usually asks whether one set of Redis, PostgreSQL and vector store servers can serve all three. Sharing is possible, but it is a deliberate isolation design, not a default. Each instance still runs its own api, worker and web containers, and the shared services become a common dependency. The question to settle first is how much isolation your environments need and who is accountable when one instance affects another. Production data and experimental indexing jobs should not compete for the same resources without a clear decision on that trade-off.
Sharing means three things. Redis holds Celery queues, cache and other transient state, so each instance needs its own key namespace, set through key prefixes and separate database numbers. PostgreSQL holds the metadata database, so each instance needs its own database or schema, with separate credentials. The vector store holds embeddings, so each instance needs its own index name prefix, so one environment cannot read or overwrite another's collections. Confirm in the environment variable reference which Redis and vector store settings accept prefixes or database numbers in the Dify version you run, and test that isolation against that version before you rely on it. Object storage and the SECRET_KEY must also be separate per instance, because a shared key lets one environment decrypt or validate another's sessions.
The Community edition is limited to a single workspace, so running separate environments is usually done with separate instances rather than with extra workspaces inside one. Dify's license also forbids using the source code to operate a multi-tenant environment, where one tenant is one workspace, without written authorization. Running several instances for your own dev, staging and production teams is a different arrangement from offering Dify to outside customers, but confirm the boundary against the LICENSE file and with your legal team before you design the layout. Ask the vendor or Chainscore Labs for a written view if your use could be read as multi-tenant.
Isolation options for shared Redis, PostgreSQL and vector stores
This table compares the main ways to separate Dify instances on shared Redis, PostgreSQL and vector stores, with the trade-off each one carries. Read each row for the isolation it gives, the work it adds for operators, and the environments it suits. Shared services are cheaper to run but share failures, memory limits and backup windows, so production usually needs stronger separation than development.
| Option | Isolation gained | Operational cost | When it fits |
|---|---|---|---|
| Separate databases on one PostgreSQL server | Separate schemas, credentials and data per instance | One server to size, patch and back up. Instances share CPU, memory and connection limits. | Development and staging with modest load and a shared maintenance window |
| Separate PostgreSQL servers | Separate failures, upgrades, backups and connection limits | Each server needs its own sizing, patching, backups and monitoring, which adds recurring cost | Production, or any instance with its own upgrade schedule or data residency needs |
| Redis key prefixes on one Redis | Namespaced keys only. Any client with access can still read or flush another instance's keys. | Low setup effort. Confirm in the environment reference whether your version exposes a prefix setting. | Low-risk environments where instances trust each other and memory pressure is low |
| Redis logical databases | Separate keyspaces by database index, but one memory limit, eviction policy and FLUSHALL scope | Confirm the database index setting for your version. Cluster-mode Redis does not offer logical databases. | Development and staging that accept shared memory and eviction behavior |
| Separate Redis instances | Separate memory, eviction, restarts and access controls | Extra processes to monitor, secure and size for each instance | Production where one instance's queue backlog must not affect another |
| Index name prefixes on one vector store | Separate collections or indexes by name. Shared cluster memory, backups and admin credentials. | Confirm the prefix setting for your chosen store. Weaviate is the default, with Qdrant, Milvus, pgvector, OpenSearch and Elasticsearch as options. | Non-sensitive test knowledge bases where one administrator manages all instances |
| Separate vector store instances | Separate storage, backups, access and capacity per instance | Separate sizing, backup jobs and upgrades for each store | Production knowledge bases with different data sensitivity or retention rules |
Which shared services each environment should share
Development is the environment where sharing carries the least risk, because its data is usually disposable and its users are a small team of builders. A single host or a shared database server can run several development Dify stacks, provided each one has its own identity in the shared services. Developers often need to break things, reset data and test migrations, so the cost of a failure here is mostly lost time rather than lost customer data.
- Application compute: Development API, worker and web containers can run on shared hosts if resource limits stop one team's indexing job from starving another.
- PostgreSQL server: A shared server can host separate databases per instance, with distinct database users that cannot read each other's data.
- Redis: A shared Redis can serve several development instances when each uses its own key prefix, so queues and caches stay separate.
- Vector store: Shared vector infrastructure is acceptable when each instance uses its own index name prefix or collection namespace.
- Secrets and encryption keys: Each development instance should generate its own
SECRET_KEYand never reuse a production value.
Before sharing, confirm with your team which of these resources developers can reset and whether any development data copies production content.
Staging exists to rehearse production, so it should match production's versions, configuration shape and migration path, but it should not share the data stores that hold real records. Sharing infrastructure with development is common and reasonable, since staging usually needs fewer resources than production. The main discipline is keeping staging isolated enough that a test migration or a runaway workflow cannot reach production identifiers.
- Database instances: Use a separate database and credentials for staging, and avoid the same database cluster as production unless the cluster is isolated by role and network path.
- Redis and queues: Give staging its own key prefix, and confirm that its Celery workers do not subscribe to queues used by production.
- Vector stores: Index names for staging should be distinct from production indexes, so a staging re-index cannot overwrite production knowledge bases.
- Object storage buckets: Separate buckets or prefixes prevent staging uploads from mixing with production files.
- Network reachability: Restrict which systems can call staging's model endpoints and external APIs, so test traffic does not consume production quotas.
As a check, confirm that no staging hostname, credential or queue name resolves to a production resource.
Production should carry the strictest dedication, because it holds the knowledge bases, conversation records, credentials and secrets that the business depends on. Sharing hardware with production is possible, but only for components that can be isolated by key, database or index, and never for the stores that define recovery. Production is also the environment where backups, restores and upgrades must be predictable, which is hard to guarantee on a platform shared with other instances.
- PostgreSQL: Keep production on its own database server or cluster, with its own backup schedule and point-in-time recovery, rather than sharing a server with test databases.
- Secrets and encryption keys: Store production
SECRET_KEYand encryption material separately, and restrict who can read them. ChangingSECRET_KEYlogs out all users, so it must never be copied into lower environments. - File storage and vector data: Use dedicated object storage buckets and dedicated vector collections or clusters with their own retention rules.
- Redis: A dedicated instance is preferred for production queues, so a flood of test jobs cannot delay indexing or workflow runs.
- Worker capacity: Give production workers reserved compute, since a misconfigured worker in a shared pool can take down the whole platform.
Before approving a shared production component, ask the vendor or platform owner which resources are isolated by key and which by server.
Sharing reduces hosting footprint, fewer servers, fewer database clusters and fewer licenses to operate, but it also widens the blast radius. A bad migration against a shared database, a misconfigured worker that consumes the wrong queue, or a vector index rebuilt under the wrong prefix can affect every environment on the same infrastructure at once. Prefixes and separate databases make mistakes less likely, but they do not stop a single shared process or host failure from reaching all instances.
- Migrations: A schema migration run against a shared database touches every instance that uses it, so migrations need an explicit target check.
- Worker misconfiguration: A worker started with the wrong environment file can process another environment's queue, so verify each worker's configuration before startup.
- Shared host failure: Compute that hosts several instances fails for all of them, which changes recovery time expectations.
- Capacity contention: Heavy indexing in development can slow production workflows if limits are not set.
- Recovery scope: Restoring one shared database can roll back data for every instance that uses it.
As a check, list every shared component and name the environments that would stop if it failed, then decide whether that outage is acceptable.
How to set up shared backing services for multiple Dify instances
This sequence turns one shared PostgreSQL, Redis and vector store into separate, verified lanes for each Dify environment, so development, staging and production cannot read or overwrite each other's data.
Inventory shared services and versions
The platform owner lists every shared backing service the instances will use: PostgreSQL, Redis, the vector store, and the object storage target, along with the exact server versions, hosts, and who administers each one. Confirm the Dify release each environment will run against the compatibility notes in the Dify docs and the GitHub releases page, since minor releases arrive roughly monthly and each environment may be on a different version. The step is done when a written inventory exists, every service has an owner, and each environment's target Dify version is recorded. Check the inventory against the Docker Compose and environment variable references before moving on, because some settings change between releases.
Create one database and role per environment
A database administrator creates a separate PostgreSQL database and a dedicated login role for each environment, for example one each for development, staging, and production. Each role gets rights only on its own database, and the default public schema privileges are reviewed so one role cannot connect to or read another environment's database. Store each password in the environment's own secrets store, never in a shared file. The check: connect as each role and confirm that a query against another environment's database is refused, and that the Dify api container for each environment reaches its own database only.
Assign distinct Redis key prefixes or database numbers
Decide whether each environment uses its own Redis logical database number or a key prefix within a shared logical database, and configure the Dify environment variables to match. Redis is used for Celery broker and result traffic, caching and related state, so a collision can cause one environment's workers to consume another's tasks. Confirm in the environment variable reference which Redis settings Dify exposes and whether they accept a database number or a prefix. The check: run a test key write from one environment and confirm it does not appear in the other environment's keyspace, and confirm each environment's worker logs show only its own queue names.
Configure distinct vector index prefixes
Set a different index or collection name prefix for each environment in the vector store configuration, so knowledge base indexes created in staging never land in the production namespace. The vector store defaults to Weaviate, but the environment reference also lists Qdrant, Milvus, pgvector, OpenSearch and Elasticsearch as options, and the prefix setting differs by backend, so confirm the exact variable for the chosen store with the Dify environment reference. Ask the vector store administrator to confirm that the chosen backend's access controls can restrict each environment's credentials to its own prefix. The check: create a test knowledge base in one environment and confirm that its index is not listed or queryable from another environment.
Generate a unique SECRET_KEY per instance
Generate a separate SECRET_KEY for each environment and store it in that environment's secrets store. The environment reference says this value signs session cookies and JWTs and encrypts OAuth credentials, so a shared key would let a session or token from one environment be accepted by another. Because changing SECRET_KEY after deployment logs out all users, set it once before first login and record where it is kept for backup and disaster recovery. The check: confirm that a session cookie issued by one environment is rejected by another, and that each environment's configuration file references a different key.
Separate storage paths and encryption material
Give each instance its own file storage location, whether a local path or a bucket or container prefix on OpenDAL, S3-compatible storage, Azure Blob or Google Cloud Storage, and keep workspace encryption keys scoped to the instance that uses them. The environment reference lists local storage and Azure Key Vault as encryption key options, so confirm which one applies and that each environment's keys are separate. Apply object storage policies so each environment's credentials can read and write only its own prefix. The check: upload a test file in one environment and confirm the other environment cannot list or fetch it using its own credentials.
Verify isolation end to end before go-live
Run a written isolation test across all shared services. For each environment, confirm that its PostgreSQL role, Redis keyspace, vector index prefix, SECRET_KEY, and storage path accept its own data and reject the others. Record the results, including refused connections and empty cross-environment listings, and keep them with the configuration inventory for later audits. Repeat this check after every Dify upgrade and after any change to the shared services, because a new release can add variables or change how prefixes are read, which the upgrade guide and .env.example will show. The step is done when every check passes and a named owner signs off each environment.
Security risks when environments share infrastructure
Development, staging and production Dify instances often share a PostgreSQL server, a Redis service and a vector store. Each shared component is a path from one environment into another, so the risks below connect to the same question: which boundary stops a credential, key or data set from crossing from one instance to the next.
Cross-environment access through shared database roles
If every instance connects to PostgreSQL with the same role, a session or script in staging can read and modify production metadata, including app configurations, conversation records and workspace settings. Give each environment its own database role with rights limited to its own database, and confirm that no role has access to another environment's schema. Check the connection settings in each instance's environment file, since a copied value is the most common way a shared role spreads.
Redis keys visible to every instance
Dify uses Redis for caching, task queues and coordination. When instances share one Redis server without distinct key prefixes or logical databases, one environment can read, overwrite or flush keys that belong to another, and a queue from staging can be picked up by a production worker. Confirm which Redis database number and key prefix each environment uses, and confirm that no environment is configured to run commands such as flush operations against a shared instance.
Vector indexes readable across prefixes
Knowledge base content lives in the vector store, and retrieval is driven by index names. If staging and production use the same vector store with index names that differ only slightly, or without a prefix, a query path can return documents from the wrong environment. Use an index name prefix per environment, verify that the vector store credentials are scoped to that prefix where the product allows it, and check that a retrieval test in staging returns only staging documents.
Shared SECRET_KEY leaking sessions
The SECRET_KEY value signs session cookies and JWTs and encrypts OAuth credentials. The environment reference states that changing it after deployment logs out all users. When two instances share a key, a session token issued by one can be accepted by the other, so a user signed in to staging may be trusted in production. Generate a separate SECRET_KEY per environment and store each in that environment's own secret store. Confirm this with the environment reference before you rotate anything in a live system, because the rotation logs out every user.
Production data copied into staging
Teams often refresh staging by copying a production database dump. That copy carries user records, conversation history, uploaded documents and any credentials stored in the database, and it moves that data into a less controlled environment. Decide in advance which data may leave production, mask or remove personal data before the copy, and restrict who can access the staging database. Confirm the refresh procedure with your data owner before the first copy is made.
Outbound requests from staging reaching internal systems
Workflow tools, HTTP nodes and knowledge sources send outbound requests from the Dify containers. The SSRF_PROXY settings route these requests through a proxy that blocks internal and private IP ranges. A staging instance that is allowed to reach a production internal address, or that runs without the proxy, can call services it should never touch. Confirm that each environment uses the SSRF proxy, that its network can reach only the endpoints that environment needs, and that network segmentation separates environments at the host or VLAN level.
Keeping several Dify versions in step on shared services
Instances that share PostgreSQL, Redis and a vector store do not have to run the same Dify version, but they do have to be upgraded with the shared services in mind. Minor releases arrive roughly monthly: the releases page lists 1.16.0 on July 17, 2026, 1.17.0 on August 25, 2026 and 1.17.1 on September 10, 2026. Development, staging and production will therefore drift apart unless someone sets a deliberate cadence, and the gap between them should be a planned decision rather than an accident of who upgraded last.
Every upgrade follows the upgrade guide for the target version on the GitHub Releases page. Take a backup before each upgrade, then compare your .env file with the current .env.example for new or renamed variables. Do this per instance, because each one has its own environment file and its own key prefixes. Confirm in the target version's notes whether it changes the database schema, the vector client or the Redis key format before you start, since those are the changes that can affect instances you have not touched yet.
The main risk is a schema or client change that is applied by an upgraded staging instance to a database that production still uses. Sequence shared-service changes so that the instance that migrates is the only one reading the state it changes, or give each environment its own database and index names. Upgrade development first, then staging, then production, and confirm that the older instances still run cleanly against any migrated shared component before moving on. Ask the vendor or check the upgrade guide for each version whether it is backward compatible with the previous one.
Limits to confirm before you share infrastructure
Several Dify instances on one Redis, PostgreSQL and vector store depend on each instance staying in its own lane. The cards below list the points that the vendor docs do not settle on their own, so each one needs a check in your environment before any shared service goes live.
Redis key prefix coverage
A key prefix only isolates instances if every place Dify writes to Redis honors it. Caches, Celery queues, locks, rate counters and pub/sub channels may each be set up differently, and some may not use the configured prefix. Map every Redis key family your version creates, then confirm that each one lands under the prefix for its instance. Run two instances against the same Redis and look for keys that appear without a prefix or that collide across instances. Treat any unprefixed key as a blocker until it is explained.
Vector store index name prefixes
Dataset collections in the vector store are named by the application, and a shared store only stays separate if those names carry a per-instance prefix. Check whether your chosen vector store provider applies the configured prefix to every collection, index and alias it creates, including those used for metadata and deletion. Confirm that dropping a dataset in one instance cannot reach a collection that belongs to another. Test this with two instances holding datasets with identical names.
Metadata database isolation model
Separate databases on one PostgreSQL server give the cleanest boundary, because migrations, roles and backups each stay per instance. Separate schemas or a shared database with shared tables offer weaker separation and can complicate restores. Ask which option your version expects, confirm that each instance can run its own migrations without touching the others, and confirm that a restore of one instance does not overwrite another. Also check whether the metadata database choice (PostgreSQL, MySQL or OceanBase) changes this answer.
Connection pool sizing across instances
Each instance opens its own pools for the API, workers and scheduled jobs, so the total connection count on the shared PostgreSQL server is the sum across all instances, not the figure for one. Add up the pool sizes for every process type in every instance, leave headroom for admin sessions, migrations and backup jobs, and compare the total with the server's connection limit. Recheck after every scaling change to worker counts, because adding workers multiplies connections.
Community workspace limit
The Community edition is free but limited to a single workspace, so running several Dify instances does not give you several workspaces inside one instance. Confirm the limit against the current pricing page before you plan, because it decides whether a team that wants separate tenants needs separate instances, a commercial edition or a different layout. Also confirm that multi-workspace features you expect from one instance are not required for your development and staging split.
Questions teams ask before running several Dify instances on shared services
Should we run one shared Dify stack or separate stacks for each environment?
For most teams, separate stacks for production and everything else is the safer default, and a shared stack is only worth considering when the environments are isolated by key prefixes, index name prefixes and separate databases. A shared stack means one upgrade window, one set of Redis and PostgreSQL capacity and one blast radius when something fails. Separate stacks cost more hardware and operator time but keep a development experiment from slowing production indexing or exposing production data. Decide this with your own risk tolerance and capacity plan, and confirm each component's isolation behavior in your environment before relying on it.
How much effort does it take to set up several Dify instances on shared services?
Effort is driven mainly by how many components you share and how much isolation you need to prove. The work includes naming and prefix conventions for Redis keys, database and schema separation in PostgreSQL, index name prefixes in the vector store, separate secrets and SECRET_KEY values per environment, and a test that confirms one instance cannot read another's data. Those tasks are one-time, while the ongoing cost is upgrades, monitoring and access reviews across every instance. Ask any partner for a written scope that lists each shared component, the isolation checks and the handover material, because that scope is what the quote follows from.
What does the work need from our internal team?
Your team needs people who own the shared services and can make decisions about them: a database administrator for PostgreSQL, someone who runs Redis and the vector store, a platform engineer for the container layer and an application owner for each Dify environment. Security or compliance staff should review secrets handling, network paths and access rules before go-live. Plan time for these people during design and validation, since most delays come from waiting on approvals or on access to shared systems rather than from configuration itself.
Can our own team set up multiple Dify instances without outside help?
Yes, a capable platform team can do it in-house, provided it has run Docker Compose services, PostgreSQL and Redis in production and can read the Dify environment reference closely. The hard parts are not the first deployment but keeping environments isolated through upgrades, backups and restores. Dify releases arrive roughly monthly, as shown by the 1.16.0, 1.17.0 and 1.17.1 release dates on the GitHub releases page, so the team must keep up with changes and check new variables after each upgrade. Many organizations split the work: internal staff own operations while an outside team handles the initial design and validation.
How long does it take to validate a multi-instance Dify setup?
Validation time depends on the number of environments, the components shared between them and how many isolation tests you require. Useful checks include confirming that separate Redis key prefixes do not collide, that each instance writes to its own PostgreSQL database, that index names in the vector store stay separate and that a restore from one instance does not touch another. No supplied figure sets a duration, so ask any provider to state the validation steps and the expected number of working days in the written scope, and confirm the schedule against your change windows.
How does the Community license affect running several environments?
The Dify Open Source License prohibits using the source code to operate a multi-tenant environment, where one tenant is one workspace, without written authorization. The self-hosted Community edition is also limited to a single workspace according to the pricing page. Running development, staging and production as separate instances, each with one workspace, is a different question from multi-tenancy, but your legal team should read the LICENSE file and the pricing terms before you design the layout. If you need several workspaces inside one deployment, the Enterprise edition is the option the pricing page lists for that, with a custom quote.
Which quote details should we ask for before committing to shared Dify services?
Ask for a quote built from a written scope that names every shared component, each environment, the isolation tests, the backup and restore checks, the handover documents and any post-go-live support. Confirm who pays for hardware, cloud accounts and licenses, since those costs are separate from services. Ask which items are one-time and which recur, such as upgrade reviews and monitoring. Fixed-scope engagements usually agree deliverables and timelines in writing before work starts, so make sure the quote states both.
How DeployOpen scopes shared Dify environments
DeployOpen plans running several Dify instances on shared Redis, PostgreSQL and vector store infrastructure as a written, fixed-scope engagement. The work covers key prefixes, index name prefixes, separate databases, isolation tests and a handover your team can operate.
How we scope
- We inventory which instances share Redis, PostgreSQL and vector stores, and which must stay fully separate.
- We agree the naming scheme for Redis key prefixes, vector index prefixes and database names in writing.
- Engagements are fixed-scope, so the scope document states which instances, services and environments are covered.
How we build
- We configure each instance with its own database, Redis prefix and vector index prefix, then verify isolation.
- We test that a development workload cannot read, write or delete staging or production data in shared services.
- We align secrets, SECRET_KEY values and storage paths per environment so no instance reuses another's credentials.
How we hand over
- We document the prefix and database naming standard so new instances follow the same pattern.
- We provide a runbook for adding an instance, retiring one and checking isolation after a version upgrade.
- We walk your platform and security teams through the configuration and the isolation test results.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer, covering new instances and releases.
- Retainer work can re-run isolation checks after each Dify minor release, which arrive roughly monthly.
- Your team confirms supported service versions and capacity limits with the vendor before each change window.
