Dify multimodal knowledge bases with images: when teams need them
Some teams keep their most important knowledge in places a text-only index cannot read. Policies rely on flow diagrams, equipment manuals use labeled figures, and spreadsheets carry charts or pasted screenshots. If the knowledge base only sees the words, an assistant can answer correctly from the surrounding paragraphs and still miss the figure that settles the question. Dify can index images alongside text when the chosen models and document types support it. According to the Dify docs, DOCX and XLSX files have their embedded images extracted automatically during indexing, and the result depends on the embedding and rerank models you select, since those models must be able to represent images. Confirm current support for each document type in the Dify docs for your version before you design around it.
The Community edition is free for self-hosting and limited to a single workspace, so a team with one knowledge base owner may not need the Enterprise edition for this work. Dify is released under the Dify Open Source License, a modified Apache License 2.0 that forbids using the source code to run a multi-tenant environment without written authorization and forbids removing or changing the logo or copyright information in the console. The main cost of multimodal indexing is not the license. It is the infrastructure and the model calls for image embedding and reranking, which are usually priced differently from text-only calls. Ask your model provider for image-specific pricing before you index a large document set.
Which configuration options change image retrieval
This table compares the settings that decide whether images in a Dify knowledge base can be found and returned. Read each row as a decision to make before loading files, because several choices are difficult to change later. The main takeaway is that the embedding model and index method determine whether images are searchable at all, while rerank and chunking mode refine what reaches the model.
| Decision | Option | Effect on cost and retrieval | When it fits |
|---|---|---|---|
| Document type | DOCX and XLSX files | Embedded images are extracted automatically. Confirm which image types and placements the extractor captures in your test files. | Office files where diagrams, screenshots or charts carry the answer |
| Document type | PDF with an extraction plugin | Requires choosing and testing a plugin. Extraction quality varies, so output must be checked before indexing. | Figure-heavy or scanned PDFs where text extraction alone misses content |
| Embedding model | Text-only embedding | Lower indexing cost. Images are searchable only through nearby text, captions and alt text. | Image-light content where captions already describe the figures |
| Embedding model | Multimodal embedding | Represents image content directly. Adds model cost and requires provider and vector store compatibility to be confirmed. | Diagrams, scanned forms or charts where the visual content is the answer |
| Rerank model | No rerank model | Lower query cost and latency. Final order depends entirely on first-stage retrieval. | Small knowledge bases where first-stage results pass retrieval tests |
| Rerank model | Multimodal rerank model | Can reorder image-bearing chunks more accurately. Adds per-query cost and needs support confirmed in your deployment. | Mixed text and image results where ordering errors show up in tests |
| Chunking mode | General mode | Simpler segments. A figure can be separated from the paragraph that explains it, so delimiters and length need tuning. | Short, self-contained pages or slides where each image sits next to its caption |
| Chunking mode | Parent-child mode | Larger parent context keeps images with their explanation. Stores more text, which raises indexing cost. | Long reports where a figure's meaning depends on the surrounding section |
How to set up a multimodal knowledge base in Dify
This sequence takes a self-hosted Dify instance from version check to a tested knowledge base that returns image-derived content from DOCX, XLSX and PDF files. Each step ends with a check so problems surface before the knowledge base reaches an app.
Confirm the Dify version and Compose prerequisites
The platform engineer records the running Dify version and compares it with the release notes on the Dify GitHub releases page before changing any settings. Confirm the host meets the documented minimums of at least 2 CPU cores, 4 GB of RAM and Docker Compose 2.24.0 or later, and that the Compose stack includes the api, worker, plugin daemon, PostgreSQL, Redis and Weaviate services. Check the version's release notes for any changes to extraction or vector store behavior that affect images. Done when the version, host sizing and Compose version are written into the change record and the stack reports all services healthy.
Connect multimodal embedding and rerank models
A workspace administrator adds the model provider credentials in the model provider settings and selects an embedding model and a rerank model that the team has confirmed accept image input. Confirm capability with the provider's documentation, since Dify lists models but image support depends on the specific model and provider plugin. Store the API keys through the console rather than in shared configuration files. Done when a test call to each model returns a successful response and the selected models show the expected capabilities in the settings screen.
Create the knowledge base with high quality indexing
The knowledge base owner creates a new knowledge base, selects the high quality indexing method, and binds the multimodal embedding model chosen in the previous step. Choose the chunk mode before any document is uploaded, because it cannot be changed later without re-indexing. Set the retrieval settings to use the rerank model so that image-derived text is reranked alongside surrounding text. Done when the knowledge base appears in the workspace with the chosen embedding and rerank settings visible on its configuration page.
Prepare a labeled test set with known images
The content owner assembles a small set of DOCX, XLSX and PDF files that each contain images with a known, written description, such as a labeled chart, a diagram with a unique part number or a scanned table. Record for each image the exact question that should retrieve it and the expected answer. Include at least one PDF with an image and one with an image-only page so the extraction plugin path is tested. Done when a test sheet lists every file, the image it contains, and the question and expected answer for each.
Upload the test files and confirm indexing
The knowledge base owner uploads the test set and waits for indexing to finish, keeping batches within the default 15 MB and 5-file limits described in the file upload documentation. Review each document's segments in the console to confirm that text extracted from DOCX and XLSX images, and from PDF pages processed by the extraction plugin, appears as chunk content. Check the attachment count on each chunk against the documented per-chunk limits. Done when every test file shows indexed status and each known image has at least one chunk containing its described content or an attachment reference.
Run retrieval tests against known questions
The evaluator opens the retrieval testing view and runs each test question from the sheet, recording the returned chunks, their scores and whether the image content appears in the top results. Compare results with and without the rerank model enabled, and note the TopK and score threshold values used. Treat a missing image result as a signal to check extraction and attachment handling before blaming the embedding model. Done when every question has a recorded result and the team can state which images were retrieved, which were missed and which settings were in use.
Record settings and sign off the baseline
The platform owner saves the knowledge base settings, model selections, plugin versions and test results as the baseline for this deployment. Store the test sheet and results with the change record so that later upgrades can be checked against the same questions. Confirm with the security owner that the uploaded test files contain no production data and that access to the knowledge base follows the team's permission model. Done when the baseline record is complete, the test set is stored, and the reviewer has accepted the results or logged the open misses with owners.
How PDF image extraction differs from DOCX and XLSX
Word and Excel files are read by the extraction path that ships with Dify, so you do not add a marketplace component to pull images from these files into the index. The work sits in what the extractor finds and how the resulting chunks are shaped. Because this path belongs to the core product, its behavior changes when you upgrade Dify itself, which makes it a question for your upgrade testing as much as your document design.
- No marketplace dependency: The built-in path runs with the core services, so there is no separate plugin owner, package or approval step for Office files.
- Embedded images: Test the charts, screenshots and scanned tables your teams store inside Word and Excel, since results depend on how each image is embedded in the file.
- Spreadsheet layout: Sheets with merged cells, floating images or dense layouts can produce chunks that read poorly, so review a sample of output from each template.
- Upgrade regression: Rerun the same sample set after each Dify upgrade, because a change in the built-in path affects every Office file in the knowledge base at once.
Confirm with a test workspace which image types from your own templates survive extraction before you load the full document set.
PDFs do not follow the Office path. Image handling for PDFs is delivered through extraction plugins from the Dify marketplace, so the results depend on the plugin you choose as much as on Dify itself. Each plugin is a separate component with its own maintainer, release cadence and supported features. Do not assume that one plugin's output matches another's, and confirm the plugin name and version you plan to use before scoping the build.
- Separate component: The plugin is installed, updated and removed independently of the Dify release, so it needs its own lifecycle record.
- Feature coverage: Confirm whether it extracts embedded images, scanned pages, tables or all three, and how it links each image to its surrounding text.
- Output shape: Check what the plugin passes to indexing, because chunk boundaries and attachment behavior depend on that output.
- Documented limits: Ask the maintainer for documented limits on page count, file size and image formats, and test against them.
Run a PDF set that includes scanned pages and image-heavy layouts through the chosen plugin before you commit to it for production content.
A plugin processes document content, so installing one is a trust decision as well as a feature decision. PDFs in regulated organizations often contain personal, financial or clinical records, which makes the plugin's data path a security question for your data owners. Document who approved the plugin, where it came from and what it can see during extraction before anyone installs it in a production workspace.
- Source record: Record the publisher, the package origin and the license for the exact plugin version you approve.
- Data exposure: Determine whether the plugin sends document text or images to any external service during extraction. If it does, that outbound flow needs approval under your data policy.
- Outbound controls: Dify's SSRF proxy settings route outbound requests through a proxy that blocks internal and private IP ranges. Confirm the plugin works with that path, and treat any exception as a logged, approved change.
- Change control: Pin the approved version and require review before an update is installed in production.
- Installation rights: Limit who in the workspace can install or update plugins.
Ask the plugin's publisher in writing where extracted content is processed, where it is stored and how long it is kept.
The plugin daemon is one of the core services that Docker Compose starts alongside the api and worker services. If it fails or falls out of step with the Dify version, PDF extraction can break while Office files keep indexing, so operators need to watch both paths. Dify minor releases arrive roughly monthly, and each one can change plugin compatibility, so plugin versions belong in the upgrade plan rather than in a separate maintenance cycle.
- Daemon health: Include the plugin daemon container in health checks and restart monitoring, and keep its logs for failed extraction investigations.
- Compatibility record: Log which plugin version works with which Dify version, and confirm support in the plugin's release notes before you upgrade Dify. The Dify releases page is the reference for Dify versions.
- Upgrade order: Back up before every upgrade, follow the upgrade guide for the target version and check
.env.examplefor new variables. - Fixed test set: Keep a stable set of PDFs and Office files, and rerun extraction after each plugin or Dify change.
- Rollback path: Keep the previous approved plugin package available so the team can revert if extraction quality drops.
Confirm the exact plugin name, version and supported Dify release with the maintainer before you schedule the build.
Security and data handling for image content
Image content carries risks that text-only knowledge bases do not, because a picture can hold personal or regulated data that never appears in the surrounding text. The six cards below cover what enters the index, what the extraction path can reach, and how credentials, access and backups are scoped around it.
Personal data inside images
Scanned forms, screenshots, ID cards and charts pasted into DOCX or XLSX files can carry personal or regulated data. Once extracted and embedded, that content is searchable by anyone who can query the knowledge base. Before ingestion, decide which document sources may contribute images, and run a review of sample extractions to confirm that identifiers embedded in pictures are not reaching a shared index.
Outbound requests from extraction plugins
Extraction plugins and model calls can make outbound requests while processing a file, so a plugin you install is part of your network boundary. Review each plugin's endpoints, confirm which models or services receive image data, and decide which providers are approved before any plugin reads regulated documents. Record that review as part of the plugin trust decision.
SSRF protection through the SSRF proxy
The environment reference says SSRF_PROXY settings route outbound requests through a proxy that blocks internal and private IP ranges. Confirm that the proxy is enabled in your deployment and that image-related fetches pass through it, since an extraction path that reaches internal addresses would expose services that were never meant to be indexed. Test with a controlled internal address before go-live.
Model credential handling
Multimodal embedding and rerank models are configured with API keys or endpoint credentials that the platform stores. The SECRET_KEY signs session cookies and JWTs and encrypts OAuth credentials, and changing it after deployment logs out all users, so plan its generation and storage before the first run. Keep provider keys in your secret manager, restrict who can edit model providers, and rotate keys on a schedule you document.
Access scoping per knowledge base
Each knowledge base is a separate access boundary, so sensitive image sets belong in their own knowledge bases rather than in a shared collection filtered by metadata. Decide who can create, view and use each one, and bind API keys to the specific knowledge base they serve. Ask your team to confirm how access works in your Dify version before you assign image-heavy sources to a broad group.
Backups of files and vector data
Uploaded source files, extracted images and vector data live in different stores. File storage can be local, S3-compatible or cloud object storage, and the vector store can be Weaviate, Qdrant or another supported option. Back up the metadata database, file storage and vector store together so a restore produces consistent knowledge bases, and test a restore before you rely on the procedure.
Questions buyers ask about Dify image knowledge bases
How much does a Dify image knowledge base implementation cost?
DeployOpen does not publish a price list. A quote follows from a written scope, and most engagements are fixed-scope. The software is not the main variable: the self-hosted Community edition is free and limited to a single workspace, according to the Dify pricing page, while the Enterprise edition is custom priced. The cost components are DeployOpen's fixed-scope work (extraction setup, model configuration, retrieval testing and documentation), the model usage you pay your providers for multimodal embedding, rerank and generation calls at indexing and query time, any extraction plugin work, and the infrastructure you run. Model usage recurs with document volume and query traffic. Ask the quote to separate one-time work from recurring costs and to name which model calls it assumes.
How long does setup take, and what drives the duration?
No duration has been established for this work, so the written scope should state the timeline. The main drivers are the document types you index (plain text, DOCX, XLSX and PDFs with embedded images each need different handling), the volume and image density of the corpus, whether a PDF extraction plugin must be selected, built or tested, how many knowledge bases and access boundaries you need, and how quickly your team reviews retrieval results. Test effort is often the largest variable, because image answers must be checked against real questions. Ask for the timeline to list each milestone and the dependencies on your model endpoints and sample documents.
What must our team provide before implementation starts?
Your team needs to supply four things before build work starts:
Can our internal team build an image knowledge base without a partner?
Yes, if your engineers can operate Docker Compose or Kubernetes, manage secrets and read Dify's configuration reference. The Community edition runs anywhere Docker runs, and the Docker Compose deployment requires Docker Compose 2.24.0 or later, at least 2 CPU cores and 4 GB of RAM, according to the Dify self-hosting docs. The harder work is less about installation and more about choosing multimodal embedding and rerank models, validating image extraction on your documents, tuning retrieval and proving that data handling meets your policy. Confirm those skills in your team before committing, and ask a partner to scope only the parts you cannot cover in-house.
Do images inside PDFs work out of the box in Dify?
Confirm this on your Dify version before you plan around it. Dify automatically extracts images from DOCX and XLSX files, but PDF handling depends on the extraction approach your version uses, and some setups rely on an extraction plugin. Run a sample set that includes scanned pages, diagrams, tables and images with captions, then check whether the images appear as retrievable content and whether the surrounding text stays attached to them. Record the results per file type. If PDF images are not extracted as expected, the decision is between a plugin, a preprocessing step and converting the source documents, and each has a different cost and maintenance profile.
What changes when we upgrade Dify to a newer version?
Upgrades are a routine operation, but image indexing needs its own regression check. Dify minor releases arrive roughly monthly, for example 1.17.0 on August 25, 2026 and 1.17.1 on September 10, 2026, according to the Dify releases page. Follow the upgrade guide for the target version, take a backup before every upgrade, and compare your .env file with .env.example for new variables. After upgrading, re-run your image test set and retrieval questions, because extraction behavior, plugin compatibility and model parameters can change. Larger version jumps may need a separate migration plan, so read the release notes before scheduling.
Does sending images to a model provider change our data handling?
Yes, when the multimodal embedding, rerank or generation model is hosted outside your infrastructure. Image content leaves your environment in the same way text does, so the provider, region, retention terms and contract need review before indexing sensitive documents. Dify's SSRF_PROXY settings route outbound requests through a proxy that blocks internal and private IP ranges, which helps control where requests go, and SECRET_KEY and workspace encryption settings protect stored credentials. Ask your security team to list which document sets may be sent to which endpoints, and whether a self-hosted model is required for particular collections.
What to confirm with your vendor and your own environment
Start by confirming the one thing that decides whether image indexing works at all: whether the model you plan to use is multimodal. Image content only reaches the index when the embedding model and, where you use it, the rerank model accept image input in your Dify version. Ask the model provider or plugin author for the exact model identifier, the input types it accepts and the release in which that support appeared, then test it on a copy of your own knowledge base rather than trusting a model list. Record the Dify version, plugin version and model name together, because any one of them can change the result.
Next, confirm plugin compatibility with the target Dify release. Dify moved model and extraction capabilities into plugins, so a model plugin or extraction plugin for PDFs may declare support for a different release range than the platform you run. Check the plugin's declared compatibility against the version you are deploying, and check the release notes for your target version before any upgrade. Release minor versions arrive roughly monthly according to the Dify release history on the Dify releases page, so repeat these version checks at every upgrade, not only at the first build. A plugin that worked on your current release can break on the next one.
Finally, test retrieval on representative documents and settle the constraints that sit outside Dify. Build a small set of real DOCX, XLSX and PDF files that include the diagrams, scanned pages and tables your users depend on, then ask questions whose answers exist only in the images and check whether the retrieved chunks and the attached images are the ones you expect. Involve your compliance lead and data protection owner before this test, since images can carry personal data, and confirm where the embedding and rerank calls go, which region hosts the vector store and object storage, and whether your data residency rules allow that path. Document the answers so that the evidence survives the next upgrade.
How DeployOpen works on this
DeployOpen scopes, builds, tests and hands over Dify multimodal knowledge bases inside your own infrastructure. The work starts with which image and document types matter, which models will embed and rerank them, and how the attachments are checked before anything is indexed.
How we scope
- Inventory the document types with embedded images, such as DOCX and XLSX files and PDFs, and confirm which need extraction plugins.
- Agree in writing which multimodal embedding and rerank models will be used and where they run.
- Define attachment expectations per chunk, file size limits and which knowledge bases will hold sensitive images.
How we build
- Configure the knowledge bases, indexing method and retrieval settings to match the agreed document set.
- Connect the chosen extraction plugins and model providers, with credentials stored as secrets in your environment.
- Run retrieval tests with real questions that depend on image content, and tune settings against the results.
How we hand over
- Document the configuration, model choices, chunk attachment limits and the steps for adding new document types.
- Walk through the retrieval test set so your team can repeat the checks after changes.
- Export configuration and record the backup and restore steps needed before future Dify upgrades.
How we support
- Fixed-scope engagements are agreed in writing before work starts, with deliverables and timelines stated in the scope.
- Ongoing support, upgrades and continuous testing are available as a retainer, including re-testing after Dify releases.
- Ask for a quote based on your written scope, which should list document volumes, model endpoints and extraction plugins.
