Dify chunking: which mode to choose before indexing locks it in
A team loading policies, manuals or contracts into a Dify knowledge base has to choose a chunk mode before indexing starts, and that choice shapes retrieval quality. Chunking is the step where Dify splits each document into segments, and those segments are what the model later searches and cites. Changing the mode after indexing means re-processing the documents, so treat the decision as locked in for practical purposes. The Dify documentation describes how each mode behaves, but exact field names, labels and limits change between releases, so confirm them in the version you run before you load production files.
In general mode, the settings that matter most are the delimiter, the maximum chunk length and the overlap between neighboring chunks. Delimiters decide where a split is allowed, so a policy with numbered clauses behaves differently from a manual that uses headings and short paragraphs. Parent-child mode separates the text used for matching from the larger text passed to the model, and the parent scope decides whether that larger block is a paragraph or the full document. Full-document parents suit short, self-contained files, but they are subject to a token ceiling for each document, so long contracts may need paragraph parents. Confirm that ceiling in your version's documentation.
Before you commit, pick a representative sample of each document type, index it under each candidate mode in a test knowledge base, and check the chunk previews for split points inside clauses, tables or defined terms. Write down the chosen delimiter, length, overlap and parent scope, and keep the original source files so a re-index is possible. The sibling sections on this page cover each setting in more depth, and the retrieval testing topic describes how to tell a chunking problem from a model problem once real questions are running.
How general and parent-child modes differ in practice
General mode splits each document into chunks of a set maximum length, using delimiters you choose and an overlap between neighboring chunks. Retrieval works on a single level: the chunk that matches the question is the text the model receives. This keeps the index simple and makes stored text easy to inspect when you debug answers. It suits documents with uniform structure, such as short policies, FAQ-style answers, product sheets and ticket summaries, where each section answers a question on its own.
- Delimiter: Choose a separator that follows the document's structure, such as a heading marker or a blank line. A delimiter that rarely appears in the text produces oversized chunks, so test it on a sample file first.
- Maximum length: Sets how much text a match carries into the prompt. Large chunks dilute the match with unrelated text, while small chunks can cut a sentence away from the condition it depends on.
- Overlap: Repeats text at chunk boundaries so a fact that spans a split can still be retrieved. More overlap increases index size and the number of near-duplicate matches.
- Retrieval effect: The matched chunk is the entire context. Precision is high for self-contained passages, but a definition in one chunk and its exception in the next may never reach the model together.
Before indexing, review the chunk preview for a sample of documents, since chunk boundaries are exactly what the model will see.
In parent-child mode, a document is divided into parent paragraphs and smaller child chunks. Search matches the child chunks, which gives precise hits, and the system returns the parent paragraph so the model has the surrounding context. This suits long, structured documents such as policies with numbered clauses, technical manuals and contracts, where a short clause is meaningless without its section. The retrieval effect is narrow matching combined with broader context, which often improves answers to specific questions.
- Child chunks: These are the search units. Smaller children help the match land on specific terms, clause numbers and product names.
- Parent paragraphs: These are the returned units and are shaped by the document's own paragraphs. Confirm in your release how parent boundaries are derived for your file types.
- Token cost: Larger parents raise the number of tokens sent to the model for each match, which affects both model cost and response time.
- Repeated parents: Several child matches from the same paragraph can return the same parent. Check result variety in retrieval tests so the context does not repeat one section.
Ask the team to confirm how separators apply to each level of the hierarchy, because a mismatch between structure and separators is the most common cause of poor parent boundaries.
The full-document option uses the whole file as the parent. When a child chunk matches, the model receives the entire document. This suits short, self-contained files where the answer depends on several sections at once, such as a single-page procedure, a form with instructions or a short standard operating procedure. Full-document parents are limited by size: Dify applies a token limit to this option, so documents above it cannot use it. Confirm the current figure and behavior in the release you run before designing a corpus around it, because the limit is the first gate for this choice.
- Suitable files: Short documents whose meaning depends on the whole text, not on one passage.
- Size check: Test the longest files in the corpus against the limit for your version. Files over the limit will not qualify for this option.
- Token cost: Every match sends the full document, so cost grows with the number of results returned and the length of each conversation turn.
- Mixed corpora: A knowledge base holding both very short and much longer files may need separate knowledge bases, since file size decides the usable mode.
Run a long document through a test before committing a corpus to this mode, and record the token count the app sends for a typical question.
The practical decision comes down to matching each document type to a mode and proving the choice with questions users will actually ask. Plan the mode for each knowledge base before the first upload, and confirm with your Dify release whether the chunk mode can be changed on a knowledge base that already contains documents. Treat the choice as fixed until you have verified it, because re-indexing a large corpus takes time and can change the cost of every later query.
- Group by document type: Put uniform, short files in general mode and structured, long files in parent-child mode, keeping them in separate knowledge bases where the mix is wide.
- Build a test set: Write questions from support logs or subject experts, including ones that span sections, and record which chunks each mode returns.
- Compare outcomes: Look at whether the model receives the condition, exception or definition it needs, not only whether a document was found.
- Watch the signals: Missing conditions, repeated context and answers that quote the wrong section usually point to chunk boundaries rather than the model.
- Document the decision: Record the mode, delimiter, length, overlap and the test results so the setting can be reviewed at each upgrade.
Before you lock the setting, confirm the current full-document limit and the mode-change rules in the release you run.
Chunk settings and when each fits
This table compares the chunk settings you choose in a Dify knowledge base and the document types each one suits. Read each row for what it controls, where it fits, and what it costs in retrieval precision, context size and index size. The main takeaway is that delimiter and length choices shape every later retrieval result, so confirm them against real documents before indexing a large set.
| Setting | What it controls and suits | Main trade-off |
|---|---|---|
| Delimiter (general mode) | Character or pattern where Dify splits text. Suits policy text with clear section breaks and FAQs with consistent separators | A poor delimiter cuts sentences or tables mid-thought, lowering precision |
| Maximum chunk length (general mode) | Upper size of each indexed segment. Suits short, self-contained passages such as clauses or FAQ answers | Shorter chunks sharpen matches but lose surrounding context, while longer ones dilute relevance |
| Chunk overlap (general mode) | Text repeated between neighboring chunks so a sentence split at a boundary still appears whole. Suits contracts and manuals with long cross-referenced clauses | Overlap raises index size and can return near-duplicate results that consume prompt space |
| Paragraph parent (parent-child mode) | Small child chunks are matched, then the surrounding paragraph is passed to the model. Suits long manuals and structured policy text with paragraph-level meaning | Richer context per answer, but more text sent to the model and a larger index to maintain |
| Full document parent (parent-child mode) | Child chunks are matched, then the whole document is passed as context. Suits short, self-contained documents such as one-page procedures or single FAQs | Limited to documents within the 10,000-token full document limit, and large documents exceed it |
| Tables, code and structured data | Rows, functions and table blocks split badly under generic rules. Suits content that should be extracted and chunked by structure, reviewed with retrieval testing | Generic delimiters may break tables or code, so confirm the handling of each format before indexing |
| Preprocessing cleanup | Removes whitespace, URLs or email addresses before splitting. Suits scanned exports, web pages and emails with repeated boilerplate | Cleanup can delete content users later search for, so confirm which patterns are safe to remove |
How to set up and test chunking before indexing
This sequence turns the chunk mode choice into a tested decision, so the mode, delimiters and length rules are proven on real questions before the full corpus is indexed and locked in.
Inventory document types and sizes
List every document source that will go into the knowledge base, grouped by format (PDF, DOCX, Markdown, HTML exports, spreadsheets), and record typical length, heading structure and how often each group changes. Note which documents have long sections where a single answer depends on surrounding text, such as policies or contracts, and which are short, self-contained entries such as FAQs. The inventory is done when every source group has a row with its format, rough size and structure, and a named owner who can answer questions about it.
Choose a chunk mode per knowledge base
Decide between general mode and parent-child mode for each knowledge base, not for the whole workspace. General mode splits text into segments that are retrieved directly. Parent-child mode retrieves smaller child chunks but returns larger parent blocks, either paragraph-level or full-document, to the model. Check the full-document limit of 10,000 tokens against your longest documents before choosing full-document parents, and record the reason for each choice. Done when each knowledge base has a written mode decision that the platform owner and the application owner have both signed off.
Set delimiter and length rules on a sample set
Pick a representative sample of documents from each group and configure the delimiter, maximum chunk length and overlap for the chosen mode. In general mode, set these directly. In parent-child mode, set the child and parent rules separately. Preview the resulting segments in the console and read them, checking for tables split across chunks, headings separated from their content, and boilerplate repeated in every segment. Done when the sample segments look coherent to a reviewer who did not write the rules, and the settings are recorded in a configuration note.
Index a pilot knowledge base
Load the sample set into a separate pilot knowledge base using the settings from the previous step, and let indexing finish without errors. Confirm the status of every document, and check for files that failed extraction or exceeded the size limits in your release. Keep the pilot isolated from production apps and API keys. Done when every pilot document shows a completed status and any failures have a recorded cause and fix.
Run retrieval tests with real questions
Collect a set of real user questions with the source passage that answers each one, ideally drawn from support tickets, search logs or subject-matter experts. Run them through the retrieval testing feature and record which returned segments contain the answer. Include questions that span two sections of a document, and questions where the answer sits deep in a long section. Done when every question has a recorded result, a pass or fail, and a note on the failure type.
Check that returned parents contain the answer
For parent-child pilots, verify that the parent block returned for each question contains the full answer, not only the matched child. If the parent is too narrow, the model receives partial context. If it is too broad, cost rises and unrelated text dilutes the answer. Compare results across the two mode settings if both were tested. Failures that persist after adjusting length rules usually point to chunking, not the model, so record them for the tuning review. Done when every failing question has been classified and the settings have been adjusted or confirmed.
Index the full corpus only after the pilot passes
Once the pilot results are accepted, apply the settings to the production knowledge base and index the full corpus. Confirm in the release you run whether changing the chunk mode or the segmentation settings on an existing knowledge base requires re-indexing, because the answer determines whether the pilot results are reusable without changes. Keep the configuration note with the knowledge base and repeat the retrieval test set on the full index. Done when the full retrieval test set passes at the level agreed in the pilot and the settings are documented for future ingestion.
What drives chunking cost and indexing effort
The chunk mode you pick sets three cost components: embedding calls at indexing time, the size of the vector index and stored segments, and the context sent to the model at query time. In general mode, every segment is embedded once, so cost follows the total volume of text after splitting. In parent-child mode, the child chunks are what gets embedded and searched, while the parent text is returned to the model. Embedding cost therefore tracks the child count, but the prompt size tracks the parent size, which is why a large parent can raise per-query model spend even when indexing looks cheap. Confirm with your model provider how embedding is billed, and whether your embedding model is hosted or local, because that changes who pays for indexing and where the load lands.
Parent size, overlap and document count each push cost up in a different way. Larger parents and more overlap multiply the text you store and, for overlap, the number of embedded units. Document count drives both: a corpus that doubles doubles indexing work. Build a sizing sheet before you index, with the number of documents, average length after cleaning, your chosen delimiters and maximum lengths, and the number of questions you expect per day. Retrieval testing against a sample of real questions shows how many parent tokens reach the model in practice, which is the number that dominates query cost over time.
Initial indexing and the retrieval tests you run beforehand are one-time costs. Re-indexing is recurring and often larger than expected: changing the chunk mode, delimiters, maximum length or overlap in general mode means re-processing existing documents, and in some cases the knowledge base cannot be reverted to a lower-cost setting once changed, so test before you commit. Add ongoing work for document growth, new source uploads and any model swap. For resource planning, the Docker Compose requirements in the self-hosting docs set only the minimum (2 CPU cores, 4 GB of RAM, Docker Compose 2.24.0 or later), and indexing load is not covered there. Ask your team or partner to size vector storage and worker capacity against your measured corpus, not the minimum.
Data handling and operational risks of chunk choices
Chunk mode decisions affect what text is stored, returned and logged, how large the index grows and how hard a later change is to undo. The cards below cover the risks that tend to surface after a knowledge base is already live, with a control to set before indexing starts.
Parent text exposure across sensitivity levels
In parent-child mode, the parent paragraph or full document is stored and returned to the app along with the child chunk that matched. If one knowledge base mixes public policy text with restricted HR or customer records, a question answered from a child chunk can return a parent that holds far more sensitive content than the match. Keep documents at different sensitivity levels in separate knowledge bases, and test retrieval output with a restricted-document question before go-live.
Re-indexing and backups before change
Changing chunk mode, delimiters, maximum length or overlap means the stored segments no longer match the new setting, so affected documents must be re-indexed. Re-indexing rewrites embeddings and segment records, which makes a rollback slow if the change goes wrong. Take a database and file storage backup before any chunk change, and confirm with your operations team how long re-indexing takes on a document set of your size.
Vector store choice and index growth
Parent-child mode stores parent text as well as child chunks, and the vector store holds embeddings for the child segments. Storage use therefore depends on both the chunk sizes you pick and the vector database behind them. Dify defaults to Weaviate and also supports options such as Qdrant and pgvector, so confirm the store, its sizing and its backup method with your platform team before indexing a large corpus.
Scoped API keys for knowledge base writes
Teams that automate ingestion through the Knowledge API can re-chunk or replace documents with a key that has broader reach than the job needs. A key that can write to every knowledge base can change chunk settings on sensitive sets as well as ordinary ones. Bind each automation key to a single knowledge base, keep keys on the server side, and rotate them when a pipeline owner changes.
Logging of retrieved parent content
Retrieval logs and app traces can record the segments returned to the model, and in parent-child mode those segments may be full paragraphs or documents. Logs then hold more source text than the question alone would suggest, and they inherit the retention and access rules of your logging stack. Decide which log fields capture retrieved content, who can read them, and how long they are kept before the knowledge base goes into production.
Upgrade effects on indexing behavior
Dify releases arrive roughly monthly, and a version change can alter how existing segments are handled, indexed or queried, so a chunk setting that worked in one release may behave differently after an upgrade. Read the release notes for the target version, back up before upgrading, and rerun retrieval tests on a reference question set against a staging copy before changing production.
Questions teams ask before choosing a chunk mode
Which chunk mode suits policies and which suits technical manuals?
Policies usually suit general mode when each clause stands on its own, and manuals with numbered steps, warnings and tables often suit parent-child mode so answers keep their surrounding context. Short, self-contained policy statements tend to match questions well when chunks split on headings or blank lines. Manuals can lose meaning when cut at a fixed length, because a step separated from its precondition or warning may be retrieved without the context that makes it safe to follow. Parent-child mode keeps the full section as the parent while matching on smaller children. Test both modes on the document types you actually hold, and consider separate knowledge bases if one library mixes very different formats.
Can we switch chunk modes after a knowledge base is built?
Plan as if you cannot switch casually. Changing how documents are split means the content must be indexed again, which is a rebuild followed by a retest, not a setting you flip on live data. Confirm the exact behavior for your Dify version in the Dify docs and the release notes before you rely on it. Build a small pilot knowledge base with both modes first, then copy the chosen settings into production. Record which chunk, index and retrieval settings each knowledge base was built with, so a later rebuild can reproduce the result. Check file limits and index method in the same plan, since those settings change the rebuild too.
How much does a chunking project cost, and what drives it?
The Community edition of Dify is free to self-host, so the main costs are labor, infrastructure and model usage. Labor covers document review, configuration, test design, evaluation rounds and documentation. Infrastructure covers the servers and storage that run Dify and its vector store. Model usage covers the embedding calls made while indexing, which recur whenever content is re-indexed. The biggest drivers are the number and format of documents, how many knowledge bases you need, how many test rounds you run and how often content changes. Ask any partner for a fixed-scope quote that lists deliverables, assumptions about your documents, and whether re-indexing after the first test round is included.
How long does a chunking pilot take?
The duration depends on how many documents you test, how clean they are, how many test questions you have, and how many comparison rounds you need. A focused pilot covers one representative document set, both chunk modes and a defined question set with expected answers. A production rollout adds more sources, access rules, monitoring and sign-off from business owners. Your domain owners often set the pace, because they must review answers against the source text. Ask for a written timeline with milestones for sample review, build, retrieval test results and sign-off, and confirm which of those milestones depend on your team's availability.
What must our team provide for a chunking project?
You need to provide a representative set of documents, including the awkward formats such as scanned PDFs, tables and long procedures. You also need test questions with expected answers written by people who know the content, plus named owners who can approve the results. Access to a Dify instance with an administrator account, and to the embedding and chat model endpoints you plan to use, is needed for testing. Confirm early which data may leave your network and where documents are processed, because regulated content often has rules on that. Finally, agree who can view and use each knowledge base before the test data is loaded.
Can our team set up chunk modes without outside help?
A capable Dify administrator can configure both chunk modes, but the harder work is designing a fair test and judging the results. Teams with time to build a question set, log retrieved chunks and review answers with domain experts can often run the pilot in-house. An outside partner adds value when you need scoping, a repeatable test method, documentation for auditors or help with rebuilds across several knowledge bases. DeployOpen implements, customizes and documents self-hosted open-source platforms inside your own infrastructure and agrees fixed scopes in writing before work starts. Compare an internal estimate with a written scope from a partner before you decide.
How do we tell a chunking problem from a model problem?
Start with retrieval, not the answer. If the passage that contains the correct answer does not appear in the retrieved results, the problem is usually in chunking, index or retrieval settings. Signs include answers that cut off mid-procedure, retrieved chunks that lack the clause a question refers to, and results that change after re-indexing. If the right passage is retrieved but the answer is wrong, incomplete or invented, the problem is more likely in the model, the prompt or the instructions. Keep a fixed question set, log the retrieved chunks for every run, and change one setting at a time so you can attribute each change to a cause.
How DeployOpen works on this
DeployOpen scopes Dify chunking as a written decision set: which chunk mode each knowledge base uses, how documents are prepared, and how retrieval is tested before content is indexed. We build and hand over the configuration inside your own infrastructure, so your team can run and adjust it afterward.
How we scope
- Inventory document types, sizes and structure so each knowledge base gets a chunk mode that fits its content.
- Record delimiter, maximum length and overlap decisions for general mode, and paragraph or full document parent choices for parent-child mode.
- Confirm the 10,000-token full document limit against your largest documents before any knowledge base is designed around it.
How we build
- Create test knowledge bases with representative files and index them under each candidate chunk mode for comparison.
- Run retrieval tests with real questions from your users and record which chunk settings return usable context.
- Apply the agreed settings to production knowledge bases, noting that the chunk mode is locked once the knowledge base is saved.
How we hand over
- Deliver a written chunking standard covering modes, settings, document types and the reasons behind each choice.
- Document the re-indexing steps your team would need if a knowledge base must be rebuilt with a different chunk mode.
- Walk your platform and content owners through the retrieval test set so they can repeat tests after document changes.
How we support
- Scope fixed engagements in writing before work starts, with deliverables and timelines agreed in advance.
- Offer ongoing support, upgrades and retrieval testing as a retainer when your knowledge bases change over time.
- Review chunking settings during each Dify upgrade, since new minor releases arrive about monthly and should be checked against your standard.
