Migrate Dify from Weaviate to pgvector or Qdrant

Migrate Dify from Weaviate to pgvector or Qdrant: what the move involves

A self-hosted Dify whose knowledge bases sit in the bundled Weaviate can move those vectors to pgvector or Qdrant without rebuilding the applications around them. The move runs through two settings: you point VECTOR_STORE at the new database and run the documented flask vdb-migrate command to copy existing vector data across. The effort depends on three things: how much data sits in the knowledge bases, which target you choose, and how far you take validation. Ask your team to inventory every dataset and its document count before anyone picks a target, because that inventory sets the size of the copy and the checks that follow.

The target choice is mostly a question of where you want to operate the data. pgvector stores vectors inside PostgreSQL, so it suits a team that already runs, backs up and patches PostgreSQL and wants one database to govern. Qdrant is a separate service with its own deployment, storage and access model. Its installation docs call for block storage with a POSIX-compatible file system and rule out NFS and S3-style object storage, and its security docs state that self-deployed instances listen on all interfaces with no authentication until you configure it. Confirm the storage, TLS and API key setup in your environment before the first copy. Qdrant Cloud and Hybrid Cloud are alternatives if you do not want to operate the cluster yourself, and their pricing is set through Qdrant's own pricing page and sales team.

What maps cleanly from Weaviate and what does not

This table compares how vector data, dataset index settings, retrieval behavior, embedding dependencies and access controls carry across Weaviate, pgvector and Qdrant. Read each row as a question to answer in your environment before you set VECTOR_STORE. The main takeaway is that the vector move is the easy part: access controls, storage and the embedding model need explicit planning.

Area	Weaviate (current)	pgvector (PostgreSQL)	Qdrant
Vector data and collections	Vectors stored in Weaviate classes created by Dify	Vectors stored in tables inside the existing Dify PostgreSQL database	Vectors stored in Qdrant collections, sharded across nodes
Dataset index type	Stored per dataset; an existing index keeps its vector type even after VECTOR_STORE changes	Same rule applies; confirm the dataset's stored type before expecting a switch	Same rule applies; confirm each dataset's stored type after the move
Storage requirement	Runs as a bundled container in the Compose stack	Uses the PostgreSQL storage you already size and back up	Needs block storage with a POSIX file system; not NFS or S3 (Qdrant installation docs)
Metadata database coupling	Separate from the metadata database	Shares the PostgreSQL metadata database, so vector growth affects it	Separate service with its own backup and snapshot procedure
Access controls by default	Confirm the authentication and network settings of your current Weaviate deployment	Inherits PostgreSQL roles and client authentication; confirm your pg_hba rules	Not secure by default: listens on all interfaces with no authentication until configured
Embedding model dependency	Vectors come from the embedding model used at indexing time	Same dependency; a new embedding model means re-embedding the corpus	Same dependency; changing the model requires re-embedding before retrieval is comparable
Retrieval behavior	Confirm score thresholds and top-k results against a test set after cutover	Confirm similarity operator and index type match your retrieval settings	Confirm filtering and scoring results on a test set; ranking may differ
High availability	Confirm the replication setup of your current Weaviate deployment	Depends on PostgreSQL high availability; automatic failover needs external tools	Multi-node Raft clusters; at least three nodes recommended for production

Which documented commands and settings perform the vector move

The vector move depends on a small set of documented pieces: the VECTOR_STORE setting, the flask vdb-migrate command, a backup path for the source data and a target that Dify documents as tested. Each card below covers one piece and what to confirm before running it.

VECTOR_STORE sets the target

Setting VECTOR_STORE in the environment file tells Dify which vector database to use for new indexing and for the migration. Changing it alone does not move existing vectors, so the value must match the store you intend to migrate into. Check the variable's exact name and accepted values in the environment reference for your release, and confirm the connection settings for the target are present before you restart services.

flask vdb-migrate moves existing vectors

The documented migration runs flask vdb-migrate after VECTOR_STORE points at the new database. Run it in a maintenance window against a fresh backup, and read the command output for each dataset it processes. Confirm the command syntax and any options for your release in the storage and migration docs, since the flags may change between versions.

Tested targets: Qdrant, Milvus, AnalyticDB

The Dify docs list Qdrant, Milvus and AnalyticDB as tested migration targets. pgvector is a common choice for teams that want to reuse PostgreSQL, but it should be confirmed against the current Dify docs for the release you run before you plan around it. Ask your team to record the target, the release and the test result in the change ticket.

Dataset index type overrides VECTOR_STORE

A dataset that already has an index keeps its stored vector type, and that type takes precedence over the VECTOR_STORE value. This means a global change can leave some knowledge bases on the old store. Before cutover, check each dataset's index type and confirm that the migration has rewritten it, rather than assuming the global setting applied everywhere.

Qdrant snapshots as a backup path

Qdrant snapshots capture collection data on the Qdrant side, which gives you a restore point for the target and a way to roll back a failed load. Take snapshots before and after the migration and test a restore on a separate instance. Snapshots protect only Qdrant data, so keep a separate backup of the Dify PostgreSQL database and the storage volumes.

Qdrant migration tool for external sources

Qdrant's migration tool is designed to move vectors into Qdrant from other systems. It is useful when you need a copy of vectors outside the Dify migration path, such as validating counts from an exported source. Confirm which source systems it supports for your versions before you rely on it, because it does not replace the Dify command for knowledge bases.

How to sequence the vector store cutover with a parallel run

This sequence moves knowledge base vectors from Weaviate to the target store in controlled stages, so each dataset is proven against retrieval results before Weaviate is switched off. The output is a verified cutover and a Weaviate decommission record that your team can audit.

Inventory datasets and their index types
Start by listing every knowledge base in each workspace, its document count, its update frequency and the vector index type stored on each dataset. Record which datasets already carry a stored vector type, because that stored type takes precedence over the VECTOR_STORE setting. Your team or the partner runs this with read access to the Dify console or database, and the output is a table with one row per dataset and a named owner for each. Datasets that will not migrate, such as retired test bases, should be marked now so they are not carried forward by accident.
Choose the target store per dataset group
Decide whether pgvector or Qdrant fits each group of datasets, based on the database you already operate, expected collection size, filtering needs and the team that will run it. pgvector reuses PostgreSQL operations, backups and access controls. Qdrant needs block storage with a POSIX-compatible file system (not NFS or S3) and, for production, a cluster design with at least three nodes. Confirm the chosen target with your platform team and record the decision, because it sets the storage, backup and sizing work that follows.
Prove the migration on a pilot dataset
Choose one small, representative dataset and run the migration against a non-production copy of the Dify environment. Set VECTOR_STORE to the target in that copy, run flask vdb-migrate as the Dify storage and migration documentation describes, then confirm the dataset's vectors and chunk counts match the source. Document any error and the exact fix. The pilot is complete when the team can state the commands, the expected counts and the time it took to validate, using only what was observed in your environment.
Take a verified backup before the full run
Before touching production, back up the Dify PostgreSQL database, the file storage holding uploaded documents and the Weaviate data, and keep the current .env file. Restore the backup into a separate environment and confirm the knowledge bases load and answer queries. This is also the rule for every Dify upgrade, which the Docker Compose documentation says should be preceded by a backup. Check .env.example for new variables in the target version, because a new variable can change behavior after an upgrade. The step is done when a restore has been tested, not merely when a backup file exists.
Run the full migration in a maintenance window
Move the remaining datasets with flask vdb-migrate on the production environment, dataset group by dataset group, while recording counts for each group. Watch for datasets whose stored index type still points to Weaviate, since those will not follow the VECTOR_STORE change and need their own handling. Keep the source Weaviate data intact throughout. The step closes with a reconciliation report listing, for every dataset, the source count, the target count and any exceptions.
Compare retrieval during a parallel run
Run both stores side by side for a fixed evaluation set of real questions drawn from users and subject owners. For each question, compare the retrieved chunks, their order and the final answer from the workflow or agent that uses the knowledge base. Score differences with the owners of each dataset, and investigate any regression before proceeding. Note that changing the store does not change embeddings, so differences usually point to index settings, filters or metadata rather than to the vector database itself. The parallel run is complete when every dataset has an owner sign-off.
Cut over and decommission Weaviate
Switch production traffic to the target store by confirming the VECTOR_STORE setting and the stored index type for every dataset, then restart the services and repeat the spot checks. Keep Weaviate read-only and retained for a period your change board agrees on, with its backup kept. Only after that period, stop the Weaviate container, remove its volume under a documented change, and update the .env and deployment files so no stale Weaviate settings remain. The final record should show the cutover date, the sign-offs and the removal of the old store.

Why a dataset's stored index type can override VECTOR_STORE

Changing VECTOR_STORE alone may not move every knowledge base. Dify's environment variable reference says that when a dataset already has an index, the vector type stored for that dataset takes precedence over the VECTOR_STORE setting. A workspace can therefore end up with some datasets reading from Weaviate and others from pgvector or Qdrant, even after the environment file has been changed and the services restarted. Treat the stored index type as the real source of truth for each dataset, and the environment variable as the default for new indexes.

Build an inventory before you touch the environment file. For each dataset in each workspace, record its name, ID, document count, indexing technique, and the vector type it currently uses. Dify's API and database are the places to gather this, so confirm with your deployed version which dataset fields expose the stored type on your installation before you rely on a script. Keep the inventory with the backup you take before the move, because it is the baseline you will compare against after flask vdb-migrate finishes.

After the move, repeat the inventory and compare the two lists. Each dataset should report the target store, and any dataset still reporting Weaviate needs a decision: migrate it, re-index it from source documents, or document why it stays. Confirm reads as well as metadata by running a retrieval test against a known passage in one dataset per store, and check the Dify and vector store logs for connection errors. Note that the dataset-level setting is what Dify follows, so a green environment check does not prove that every knowledge base has moved.

Choosing pgvector to reuse PostgreSQL or Qdrant for scale

Choose pgvector when your organization already runs PostgreSQL for the Dify metadata database and has the skills to operate it. In that case the vector data sits in a system your team already backs up, monitors, patches and restores, so you add no new stateful service to the architecture. The trade-off is that vector search shares the resources and failure domain of your relational database, so capacity planning has to account for both workloads.

Operational footprint: One database engine, one backup regime and one upgrade calendar cover metadata and vectors.
Version support: PostgreSQL 14 supports through November 12, 2026, 15 through November 11, 2027, 16 through November 9, 2028, 17 through November 8, 2029 and 18 through November 14, 2030, according to the PostgreSQL versioning policy. Confirm which major version you run before choosing pgvector.
High availability: Hot standby replicas serve read-only queries, and synchronous or asynchronous replication is available, but PostgreSQL does not detect primary failure on its own. Plan external failover tooling.
Connection management: Dify and vector queries can exhaust connections, so confirm whether a pooler such as PgBouncer belongs in front of the database.
Authentication: PostgreSQL supports SCRAM, certificates and directory-based methods, so align the database login model with your identity policy.

Ask your database team whether the existing cluster has headroom and a tested recovery path before adding vector load to it.

What drives the cost and duration of a vector store migration

This table lists the factors that change the effort and cost of moving Dify knowledge vectors from Weaviate to pgvector or Qdrant, and whether each cost is one-time or recurring. Read it before requesting a quote, so the written scope names the datasets, embedding model, target sizing and environments that the estimate depends on.

Driver or cost	Why it changes cost or effort	What to confirm or keep down
Number of datasets and documents	Re-indexing time and validation scope grow with the number of datasets and the documents inside them	Inventory datasets, document counts and index types before scoping, and migrate in batches per environment
Re-embedding need	If the embedding model changes, every chunk must be embedded again, which adds model usage charges on top of the migration work	Keep the embedding model unchanged where possible so vectors can be moved rather than regenerated; confirm the provider's per-token charges
Target sizing and storage class	Qdrant needs block-level storage with a POSIX-compatible file system and cannot run on NFS or S3; SSD or NVMe is recommended (Qdrant installation docs)	Ask for the target's CPU, memory and disk sizing against your vector count, and confirm the storage class before procurement
Replication factor	Qdrant's replication factor stores extra copies, and a factor of 2 doubles storage (Qdrant distributed deployment docs)	Set the replication factor in the written scope and size disks for it; production clusters are recommended to use at least three nodes
Security configuration of the target	By default self-deployed Qdrant listens on all interfaces with no authentication, so API keys, TLS and network policy must be configured (Qdrant security docs)	Include API key, TLS, network restriction and audit logging setup in the scope and in the validation checklist
Validation depth	Retrieval comparison across sample queries, counts and access tests takes more effort than a row-count check alone	Agree in writing which queries, datasets and answer-quality checks will be run and who signs off
Number of environments	Each environment (test, staging, production) repeats the migration, validation and cutover, multiplying effort	Ask whether lower environments can be migrated first and reused as rehearsal for production
One-time costs	Migration build, data movement, validation and cutover are paid once per migration, and re-embedding is paid once if the model changes	Request these as a separate fixed line in the quote
Recurring costs	Hosting, storage, backups, patching and Qdrant or PostgreSQL upgrades continue after cutover; Qdrant Cloud pricing and Enterprise options are quoted separately (Qdrant pricing page)	Ask for the ongoing support and patching model, and compare self-hosted operations with any managed tier before choosing

What usually goes wrong during a vector store switch and how to prevent it

A vector store switch fails less often on the copy itself than on the state around it: which store each dataset reads from, how the new store is exposed, and whether the old data can come back. The cards below cover the failures we see most in planning, each with a check to run before cutover.

Dataset reads from the old store

A dataset that already has an index keeps its stored vector type, and that stored type takes precedence over the VECTOR_STORE setting. Changing the environment variable alone can leave some knowledge bases querying Weaviate while others query the new store. Before cutover, list every dataset and confirm its stored vector type, then confirm that retrieval test queries return results from the target store for each one.

Qdrant exposed without authentication

Qdrant's security documentation states that self-deployed instances listen on all network interfaces and have no authentication configured by default. A fresh node on a shared network can be read or written by anyone who reaches its port. Before Dify connects, bind the service to the private network segment, enable an API key, enable TLS for REST and gRPC, and confirm from a host outside the Dify network that the port is unreachable.

Qdrant on network or object storage

Qdrant needs block-level storage with a POSIX-compatible file system and does not run on NFS or S3-style object storage. A volume that looks sufficient in a test can fail or corrupt data under load on the wrong backend. Ask your infrastructure team to confirm the storage class is local or block-backed, prefer SSD or NVMe, and test a restart of the node with the collection loaded before any production data is moved.

Retrieval drift after re-embedding

If the migration re-embeds chunks with a different model or settings, answers can change even though the store is healthy. Users may notice only after a support escalation. Keep the embedding model, chunk settings and retrieval strategy unchanged during the switch, and run a fixed set of representative questions against the old and new stores, comparing the cited source chunks for each.

No verified backup before migration

The migration rewrites vector data in the target store, and the Weaviate copy is the fallback if the move fails partway. Take a snapshot of the Weaviate data volume and a PostgreSQL dump of the metadata database before running the migration command, and restore one snapshot into a scratch environment to confirm it opens. Record the backup location and the restore steps in the change ticket.

Rollback path left undefined

Teams often plan the forward migration and skip the return path. If the new store fails after users are querying it, the old setting must still work. Keep the Weaviate container and its volume intact until validation passes, document the exact VECTOR_STORE value and dataset state needed to revert, and rehearse that revert once in a non-production copy before the cutover window.

Questions buyers ask before moving Dify vectors to pgvector or Qdrant

What does it cost to move Dify vectors from Weaviate to pgvector or Qdrant?

The cost is mostly labor and infrastructure, not licenses. The open-source engines are free to self-host, so the main line items are the people who plan and run the migration, the target infrastructure, and any re-embedding of documents. Qdrant's open-source engine is licensed under Apache-2.0 and is free to self-host, while Qdrant Cloud, Hybrid Cloud and Private Cloud are priced separately. pgvector adds no license cost on an existing PostgreSQL. Ask any partner for a quote built from a written scope that lists the knowledge bases, document volume, target infrastructure, validation steps and who performs each task. Costs that recur are hosting, backups, monitoring and upgrades. Costs that are one-time are planning, the cutover and validation.

Do I have to re-embed my documents when I switch vector stores?

Not always. The vector data already stored in your datasets is what Dify migrates when you set VECTOR_STORE to the new database and run flask vdb-migrate. Re-embedding becomes necessary when you also change the embedding model, the chunking settings, or when a knowledge base has no usable stored vectors. Re-embedding costs model calls and time in proportion to document volume, so confirm which embedding model each dataset uses before you plan the work. Check the Dify storage and migration docs for the exact behavior of your Dify version before you assume either path.

Why does my migration still hit Weaviate after I change VECTOR_STORE?

A dataset that already has an index keeps its stored vector type. Dify gives that stored type precedence over the VECTOR_STORE setting, so changing the environment variable alone does not move existing datasets. Inventory each dataset's vector type before the cutover, run the migration, then confirm that the datasets now report the new store. Leaving some datasets on the old backend means you must keep the old service running, which adds risk and cost to the project.

Should I choose pgvector or Qdrant for my Dify deployment?

Choose pgvector when you already run PostgreSQL, your collection sizes are moderate, and you want one database to back up, patch and monitor. Choose Qdrant when vector volume, query load or filtering needs will grow beyond what you want to tune inside PostgreSQL, or when you want a dedicated vector service with its own scaling path. Qdrant needs block storage with a POSIX-compatible file system, SSD or NVMe is recommended, and it cannot run on NFS or S3. A production Qdrant cluster should have at least three nodes so quorum survives a node failure. Compare both options against your data volume and your team's operating skills before you decide.

What must my team provide before a vector store migration starts?

Expect to provide a current Dify version and a full inventory of knowledge bases, their embedding models and document counts. You also need access to the Dify environment file, the database and the Weaviate instance, plus a non-production copy for rehearsal. Name a decision owner for the cutover window and the rollback criteria. Finally, provide the target infrastructure, such as a PostgreSQL server or Qdrant nodes with storage that meets the Qdrant installation requirements, and access to your secrets store for the new service credentials.

How long does a Dify vector store migration take?

The duration depends on the number of datasets, the total vector count, whether any re-embedding is needed, the size of the target cluster, and how many validation cycles your change process requires. No single figure applies across deployments, so ask for a timeline in the written scope and check that it lists each phase: inventory, rehearsal, migration, validation and cutover. Ask how the plan handles datasets that fail to migrate and what rollback looks like if the target store is unavailable during the window.

Can my internal team run the migration without an outside partner?

Yes, if your team can run the documented flask vdb-migrate workflow, operate the target database or cluster, and verify retrieval results against known answers. The work is manageable for teams that already run PostgreSQL or Kubernetes and have a test environment. The risks are in planning and validation: missing a dataset that keeps its old vector type, skipping authentication on a new Qdrant instance that is insecure by default, or failing to back up before the cutover. An outside partner can take over any part of that work under a fixed-scope agreement, and DeployOpen can plan, implement and validate the migration inside your own infrastructure.

How do I confirm the migrated knowledge bases return correct results?

Validation should compare retrieval results before and after the move. Keep a fixed set of test questions with expected source documents for each knowledge base, run them against Weaviate before the cutover, then repeat them on the new store. Check dataset vector counts against the source, confirm that each dataset reports the new store type, and review failed or empty datasets individually. Qdrant's security settings also need checking, since self-deployed instances listen on all interfaces and have no authentication by default. Take a snapshot or database backup before the cutover so you can restore if validation fails.

How DeployOpen scopes and delivers a Dify vector store migration

DeployOpen starts a Dify vector store migration with a written scope that names the source and target stores, the Dify version in use, the knowledge bases in play and the environment where the work runs. Most of the cost and risk sits in decisions made before any data moves, so the scope should record which dataset indexes exist, whether any dataset carries its own stored vector type, and whether the target is pgvector on an existing PostgreSQL service or a new Qdrant deployment. The inventory step checks each dataset against the VECTOR_STORE setting, because a dataset that already has an index keeps its stored vector type regardless of the global value. Confirm the current Dify version and its upgrade notes before the scope is signed, since the migration command and its prerequisites belong to that version.

A pilot runs first, on a copy of the Dify database and a small, representative knowledge base, inside the client's own cloud account, data center or air-gapped network. The pilot measures how long re-embedding takes against the embedding model the client has approved, which is usually the largest recurring cost in the project, and it tests the flask vdb-migrate run end to end. Production work follows only after the pilot results are accepted. Throughout, DeployOpen works in the client's infrastructure, and the client's team keeps custody of credentials, backups and the decision to cut over.

Handover includes validation evidence: retrieval comparisons between old and new stores on agreed test queries, record counts per dataset, a rollback plan with the old store kept until acceptance, and runbooks for the new store's operation. Ongoing support, upgrades and continuous testing can be added through a retainer agreed in writing, which suits teams that expect Qdrant cluster changes or Dify minor releases after cutover. DeployOpen prepares this evidence for the client's own review and audit processes, but it does not issue audit reports or certifications.

Migrate Dify from Weaviate to pgvector or Qdrant