When a custom Dify model provider plugin is the right build
A custom model provider plugin is the right build when your regulated team wants an internal model gateway or a fine-tuned model to appear in Dify's model picker next to the approved providers, with the same workflow, agent and knowledge base behavior users already rely on. In Dify, a model plugin registers a provider with the platform and declares which model types it exposes, such as language, embedding, rerank or speech models, along with the credential fields an administrator fills in once per workspace. Once installed, the model shows up as a choice in apps and knowledge bases, so builders do not need to wire HTTP calls into each workflow. That consistency is the main reason to choose a plugin over a generic OpenAI-compatible endpoint, which Dify can often reach through its standard provider configuration without any code.
The choice is not only about convenience. A generic endpoint works well when one model speaks a familiar API shape and needs few extra parameters. A model plugin earns its cost when you need to define the model list and parameter rules yourself, map internal authentication to the credential schema, control which model types appear, or keep provider logic in a versioned package that your security team can review and sign. Before committing, confirm three things with your platform team: which model types you need (the supplied Dify plugin documentation should be checked for the exact type names in your target version), whether your gateway already exposes an API that a plugin can call, and whether the gateway holds the secrets so that Dify stores only the credentials it needs.
DeployOpen plans and builds these plugins inside your own infrastructure, so the gateway endpoints, credential handling and test models stay in the environments you control. The first deliverable is usually a written scope covering the model types, the credential fields, the outbound network path through Dify's SSRF proxy settings and the test plan. Ask any provider you evaluate to show how the plugin is packaged, how a new version is installed in an existing Dify instance and how the work is tested against your real gateway before it reaches production. The Dify GitHub repository is the natural starting point for confirming the plugin interfaces that apply to your release.
What a model plugin can expose in Dify
A model plugin can declare several model types, and each one reaches a different part of Dify. The cards below map each type to the features that call it, along with the provider configuration you need to plan before writing the plugin.
LLM model type
The LLM type serves chat and completion models to workflows, chatflows, agents and prompt-driven nodes. It is usually the first type an internal model gateway exposes. For each model you declare, check what the schema advertises, including streaming, tool calling, vision support and context length, because those declarations shape what the console offers. Confirm each flag against the gateway's actual behavior before publishing.
Text embedding model type
The text embedding type supplies vectors for knowledge base indexing and query-time retrieval. An embedding model fixes the vector output for the documents it indexes, so replacing it later usually means re-embedding existing content. Ask the owning team which embedding models the gateway will serve, whether they handle the languages in your corpus, and how long an older model stays available for documents already indexed.
Rerank model type
The rerank type scores retrieved passages against a query so the strongest matches reach the prompt. Knowledge retrieval settings call it, not chat generation, so a gateway can expose reranking without exposing an LLM. Confirm the rerank endpoint's input limits, such as the maximum number of passages per request, and how it returns scores, since your plugin must map those scores to what Dify expects. Test with a sample knowledge base before rollout.
Other declared model types
The plugin docs list further model types beyond these three, such as speech-to-text, text-to-speech and content moderation. Declare a type only when the gateway serves it and a Dify feature will call it, since an unused declaration adds configuration without adding function. Confirm the exact type identifiers in the Dify plugin documentation for the SDK version you build against before writing the manifest.
Provider credential schema
The provider credential schema defines the fields a workspace administrator fills in once, typically a base URL and an API key or token. Dify validates these fields when credentials are saved, so your validation call should reach a light authenticated endpoint on the gateway. Mark secret fields as secret so Dify stores them as encrypted credentials, and plan separate credential sets per workspace so one team's key is not shared with another.
Model selection in apps
Model-level parameters, such as a deployment name or default generation settings, belong in the model schema. Workflow LLM nodes, agents, knowledge bases and the prompt IDE each choose from the models your plugin registers. Before rollout, list which apps use each model, because removing or renaming a model leaves those apps needing reselection. The app management docs cover where these choices appear in the console.
Which model types and credential patterns to implement
This table compares the main decisions for a custom model provider plugin: which model types to expose, whether to write one plugin or several, how to store credentials, and whether those credentials are scoped per workspace or shared. Read each row for when the option fits and what it adds to day-two operations. The main takeaway is that broad model type coverage and shared credentials both widen the blast radius, so start with the narrowest set your teams need.
| Decision | When it fits | Operational cost |
|---|---|---|
| LLM model type only | Chat or completion endpoints from an internal gateway that serves one purpose | Smallest schema surface to build and test; embedding or rerank use cases need a later plugin change |
| Add embedding or rerank types | Knowledge bases need your internal embedding or rerank endpoint | Each type needs its own schema, test calls and retrieval checks against your index settings |
| One provider plugin | A single internal gateway exposes all models behind one endpoint and one credential set | One package to version, sign and upgrade, but every model change touches the same plugin |
| Several provider plugins | Gateways differ in owner, authentication or data class and must be separated | More packages to build, distribute and patch, and more daemon memory and version drift to watch |
| API key credential | The gateway issues long-lived keys that your platform team can rotate on a schedule | Keys must be rotated in Dify's credential store; expired keys surface as failed runs, not clear alerts |
| Gateway token credential | Your gateway issues short-lived tokens from an identity provider, so Dify holds only a client secret or refresh path | Token refresh logic lives in the plugin and must be tested against your identity provider's expiry behavior |
| Per-workspace credentials | Different business units must be billed, audited and revoked separately | Each workspace needs its own credential entry, and rotation becomes a per-workspace task |
| Shared credentials | One approved gateway identity serves all workspaces under a single policy | Rotation is simple, but one leaked secret exposes every workspace and per-team usage is harder to attribute |
How the credential schema protects keys in a workspace
The provider credential schema is the list of fields a plugin declares for a model provider, such as an endpoint URL, an API key or a gateway token. Dify renders those fields in the console when a workspace admin adds the provider, so the schema decides what a user can type and which fields are marked as secret. Keep the schema minimal: one base URL and one credential is usually enough for an internal gateway, and every extra field is another value that has to be stored, rotated and explained to the next admin. Confirm in your plugin's manifest and in a test install exactly which fields the console masks, because the masking rules come from how the schema is written, not from a platform-wide setting.
Once saved, credential values are written to the metadata database in encrypted form. Two environment settings govern that protection. SECRET_KEY signs session cookies and JWTs and also encrypts OAuth credentials, and the Dify environment reference warns that changing it after deployment logs out all users. Workspace encryption keys are a separate layer: the same reference says they can be stored locally or in Azure Key Vault. Decide where those keys live before the first provider is saved, and record that decision with the backup plan, since a database restore without the matching key material leaves stored credentials unreadable. Confirm with your own environment how the key store is configured and tested, rather than assuming the default.
The gateway call itself leaves the Dify network, and that path is controlled by the SSRF_PROXY settings. The reference describes a proxy that blocks internal and private IP ranges, so a model gateway on a private address will fail unless it is reached through an approved route. Plan that route explicitly with your network team: a named hostname, an allowlisted destination and a TLS endpoint your security team has reviewed. Do not solve a failed connection by widening the proxy for all traffic. Test the provider from the plugin's own runtime with a deliberately wrong key as well as a valid one, so you confirm both the rejection path and the stored-secret path before users see the provider.
How the model plugin build proceeds, step by step
The sequence moves from confirming the target Dify version to rolling a tested model plugin into workspaces, with a named output at each stage so the owner can sign off before the next step starts.
Confirm the target Dify version and plugin architecture
The engineering lead records the exact Dify version your instance runs and the plugin architecture that version expects, then checks the release notes for changes to model plugin interfaces before writing code. Dify's releases arrive roughly monthly, so a version pinned in the plan may be out of date by the time build starts. The output is a short compatibility record naming the Dify version, the plugin SDK or scaffold you will use, and the upgrade path if the target moves. Confirm the scaffold and SDK versions with the Dify plugin documentation rather than assuming them.
Define the models and model types to expose
The platform owner and the team that runs the gateway or fine-tuned model list every model ID, its context limits, and the Dify model types it will serve, such as LLM, text embedding or rerank. Each entry is checked against what the endpoint really supports, not what the team hopes it supports. The output is a model list approved in writing by the owner, with one line per model stating its type and the workspaces allowed to see it. Anything not on the list stays out of the plugin.
Design the credential schema
The security engineer and the plugin developer decide which values a workspace administrator must enter to connect the plugin, such as endpoint URL, API key and any region or tenant identifier. Each field is marked as secret or non-secret, and the design states who may rotate each secret and how. The output is a credential schema document and a list of outbound hosts the plugin must reach, which later feeds the SSRF proxy allowlist. Review it against your secrets policy before any code depends on it.
Implement the provider and model code
The developer writes the provider class, the model implementations for each approved type, and the credential validation logic, using the scaffold confirmed in the first step. Validation should fail clearly on a wrong key or unreachable endpoint so administrators get a useful message. The output is a source tree that builds cleanly and a changelog entry for version 0.1 of the plugin. Keep tools and other capabilities out of this package, because a model plugin cannot bundle them.
Test against a non-production instance
The team installs the package on a non-production Dify instance that matches production settings, including the plugin daemon, the SSRF proxy and the secret key configuration. They run credential validation, a chat or completion call, an embedding call where relevant, and failure cases such as timeouts and invalid keys. The output is a test record listing each model and result, and the defects fixed before release. Testing on a throwaway default install often hides proxy and network problems that appear only in production-like settings.
Package, sign and distribute the plugin
The developer builds the plugin package, assigns a version, and signs it according to your distribution policy, then places it in the internal channel you have chosen for private plugins. Signing keeps signature checks meaningful in production, so the signing key owner is named and its storage is controlled. The output is a versioned, signed package with its checksum and release notes, stored where the platform team can retrieve it. Confirm the signing and distribution options against the current Dify plugin documentation before you commit to a channel.
Roll the plugin into workspaces
The platform team installs the approved version in a staging workspace first, confirms the model list and credential form appear as designed, and then enables it in the production workspaces that the owner listed. Each workspace administrator enters their own credentials, which are stored encrypted in that workspace. The output is an installation log per workspace and a confirmed smoke test, with a rollback step documented that names the prior version. Keep a record of who approved each workspace so later audits can trace access.
What drives the effort of a model plugin build
This table lists the factors that move the size of a Dify model provider plugin build, with the reason each one matters and how to keep it contained. Read it as a checklist to bring into scoping, since the largest effort usually comes from the gateway's API behavior and the number of models rather than from Dify itself. A written list of model types and models, agreed before work starts, is the main way to keep the build bounded.
| Driver | Why it changes effort | How to keep it bounded |
|---|---|---|
| Number of model types and models | Each model type, such as LLM, text embedding or rerank, needs its own schema and test coverage, and each model needs its own parameter rules. | Agree a written list of model types and named models in the scope before build starts. |
| Gateway API compatibility | An OpenAI-compatible gateway may need little mapping, while a proprietary API needs request and response translation for every call path. | Confirm the gateway's API style and any non-standard fields with its owner before estimating. |
| Streaming behavior | Streamed token output, usage reporting at the end of a stream and error frames mid-stream each need handling and tests. | Document the exact stream format the gateway returns and test it against a sample of real responses. |
| Tool-call behavior | Models that support function calling need mapping between Dify's tool definitions and the gateway's format, and mismatches only show up in use. | State whether tool calling is in scope per model; mark unsupported models as text-only in the list. |
| Credential schema and rotation | The credential form, validation call and handling of key rotation must work for every workspace that uses the plugin. | Define the credential fields, the validation request and the rotation procedure in writing before build. |
| Testing across Dify versions | Plugin interfaces change between Dify releases, so the plugin must be checked against each version your organization runs. | Name the Dify versions in scope and the test set to run against each one. |
| Signing and distribution path | Packaging, signing and the chosen channel (local .difypkg, GitHub or Marketplace) add release steps that repeat for every version. | Choose the distribution path early and agree who holds signing keys and who approves releases. |
| Fine-tuned or internal model endpoints | Private endpoints need network paths, certificates and sometimes the SSRF proxy allowlist, which can involve other teams. | List the endpoints, hosts and owners in the scope so network changes are planned, not discovered. |
Why a model plugin cannot bundle tools
A model plugin declares models and their credentials, and nothing else. It tells Dify which model types the endpoint exposes (for example, language, embedding or rerank) and which fields a workspace must fill in before those models appear in the model list. Tool capabilities, such as functions an agent or workflow can call against an internal system, belong to a different plugin type or to a custom tool. Designing the model plugin with this boundary in mind keeps its schema small and its credential handling easy to review. Before writing any code, confirm the plugin type decision against the current Dify plugin documentation, since the model plugin contract is the part you cannot retrofit without rebuilding the package.
If your gateway needs both a model and callable functions, plan two packages with separate responsibilities: a model plugin for model exposure and credentials, and a tool plugin or custom tool for the callable operations. Give each one its own version history, since a change to a tool's parameters should not force a re-review of the model credential schema. Confirm with your platform team which credentials each package will request and whether both will run against the same internal endpoint. Shared secrets should be entered once per workspace and reviewed for scope in both packages. Use the plugin installation and distribution guidance in the Dify docs to decide how each package reaches your workspaces.
Test the pairing before rollout. Install both packages in a non-production workspace, confirm that each model type appears in the model selector with the expected credential form, and confirm that the tool appears separately under tools with its own authorization step. Check that a model is never expected to carry tool definitions, because a workflow built on that assumption will fail when moved to a workspace where only the model plugin is installed. Record which package owns each capability so that upgrades and access reviews can be scoped to one package at a time.
Questions buyers ask before building a model plugin
How much does it cost to build a custom Dify model provider plugin?
Cost follows from the written scope, and DeployOpen does not publish a price list. The main drivers are how many model types the plugin exposes, how many endpoints and authentication methods the gateway uses, how complete the gateway API specification is, and how much testing runs against real credentials. One-time costs cover design, build, testing and packaging. Recurring costs cover updates when Dify releases change the plugin interface and when the gateway itself changes. Ask any partner, including DeployOpen, for a quote that lists these items separately and states what is included in each phase.
How long does a model provider plugin take to deliver?
Duration depends on the scope agreed in writing, so no fixed timeline applies to every plugin. A gateway with one model type and a documented API is simpler than a gateway that serves chat, embedding and rerank models with per-model credential rules. Access delays also matter: waiting for test credentials, gateway documentation or a non-production Dify instance often takes longer than the coding. Ask for a timeline that ties each milestone to a client input and to a verification step in a non-production instance.
What must our team supply before a model plugin build starts?
Plan to supply four things: the gateway API specification, test credentials for a non-production environment, named owners for each model exposed through the gateway, and a non-production Dify instance that matches the production version. The specification should cover authentication, model listing, request and response formats, error codes and rate limits. Model owners confirm which models can be exposed, their context limits and their data handling rules. Without a non-production instance, testing moves into production, which the build should avoid.
Can our in-house team build a Dify model plugin without a partner?
Yes, if the team has engineers comfortable with Python and the Dify plugin development tooling, and if someone owns the gateway API. The work involves defining the model types, writing the credential schema, handling errors and writing tests against the gateway. Teams without prior plugin experience often underestimate packaging, signing and the review of how credentials are stored. A partner adds value when the team lacks capacity or needs an independent review of credential handling before production. Compare both options against the same written scope.
Will a model plugin break when we upgrade Dify?
It can. Dify's plugin architecture changed in version 1.0, and later releases can change the plugin interface, so each upgrade needs a compatibility check before production. Dify's Docker Compose upgrade guidance recommends a backup before every upgrade and a review of the environment variable template for new settings. Keep the plugin in version control, test it against each target Dify release in a non-production instance, and check the release notes on GitHub before moving production.
How do plugin signing and distribution affect production?
Signing and distribution decide whether a plugin can be installed in production and who can change it. A private plugin can be distributed as a local package, through GitHub or through the Marketplace, and each route has different controls over versions and trust. Production should keep signature checks enabled, so unsigned or modified packages are rejected. Confirm with your Dify edition which installation controls are available to you, since policy options differ between Community and Enterprise editions.
Does the plugin handle credentials securely in our environment?
The credential schema defines the fields a user enters for the gateway, such as a base URL and an API key, and Dify stores those values in its database. Dify's SECRET_KEY and workspace encryption settings protect stored secrets, so their handling belongs in the design review. Confirm where the gateway key is stored, who can read it, how it rotates and whether outbound requests to the gateway pass through the SSRF proxy with the right allowlist. Do not assume these controls are configured until they are tested.
Do Dify model plugins support bundled tools?
No. A model plugin exposes language, embedding, rerank or other model types only, and it cannot bundle tools. Tools that call internal systems need a separate tool plugin or an OpenAPI-based custom tool. Plan these as separate packages with their own credentials, versions and testing, so a change to the model gateway does not force a change to unrelated tool behavior.
Keeping a custom model plugin working across Dify upgrades
A custom model plugin is only as stable as its last test against the Dify version running in production. Dify minor releases arrive roughly monthly: 1.16.0 on July 17, 2026, 1.17.0 on August 25, 2026 and 1.17.1 on September 10, 2026, according to the Dify releases page. Treat each new minor release as a change that must be tested against your gateway plugin before it reaches users. Assign one named owner for the plugin, with a second person who can approve upgrades when the owner is unavailable. The owner decides when a release is tested, keeps the plugin's compatibility notes current and signs off the production change.
Before every upgrade, take a backup of the metadata database and the file storage, because the Docker Compose upgrade guidance recommends a backup before every upgrade. After upgrading, compare your .env file with .env.example in the new release, since new variables may need to be set for the plugin daemon or the models your plugin exposes. Read the upgrade guide for the target version before you start, and confirm with the release notes whether the plugin interface or the plugin daemon changed in that release. Record the result in a short upgrade log so the next person can see which version each plugin build was validated against.
Build a regression test for each model the plugin exposes. The test should cover every model type you declared, invoke a known prompt against each one, check the response shape and token usage, and exercise each credential field, including a deliberately invalid key to confirm the error message is clear. Run it in a staging instance that matches production's plugin settings. Monitor model call errors in production through Dify's logs and your gateway's logs, and set an alert threshold your owner agrees to. A rise in authentication, timeout or schema errors after an upgrade is the first sign that the plugin needs attention, so review those signals within the first days after each release.
How the model plugin connects to the rest of the platform
A model plugin sits between the Dify API and the model endpoint it exposes, so every connection it makes can fail, leak data or drift from what the workflow expects. The cards below cover each system the plugin touches and what to confirm before you build.
Plugin daemon and runtime
Dify runs plugins in a separate plugin daemon that the api service calls over an internal interface. Your plugin needs its manifest, provider YAML and model schemas loaded correctly, and the daemon must be able to reach your gateway from its container network. Confirm which Dify version you run, since the plugin architecture is the basis for every model plugin, and check how the daemon handles timeouts and package size limits before you assume a slow endpoint will work.
Model gateway or fine-tuned endpoint
The plugin translates Dify's invoke calls into requests your endpoint understands, including the request format, streaming behavior, token counts and error codes. Ask the team that owns the gateway which API shape it exposes, whether it returns usage figures Dify can display, and how it handles rate limits. A fine-tuned model served behind a private address also needs the SSRF proxy allowlist to permit that host, so confirm the route before testing.
Credentials and secrets handling
Credentials entered in the provider form are stored by Dify and passed to your plugin at runtime, so the credential schema defines exactly what an operator must supply, such as base URL, API key and model name. Use a gateway key scoped to one team or environment rather than a shared root key. Confirm where Dify stores encrypted values and who can read them, because the SECRET_KEY governs that encryption and changing it after deployment logs out all users.
Outbound filtering and network path
Requests from the plugin to internal model endpoints usually pass through the SSRF proxy, which blocks internal and private IP ranges by default. Your plugin may be blocked or may need an explicit allowance, depending on how the proxy is configured in your deployment. Confirm the exact hostnames, ports and TLS trust chain with the network team, and test from inside the Dify container network, not from your laptop.
Logging and LLMOps monitoring
Dify's LLMOps features record runs and usage for apps built on your model, but the plugin decides which model name, token counts and latency values are reported. Decide what your gateway logs, whether prompts and completions are retained, and how request identifiers map back to Dify apps. Confirm with the security team what may be logged, because prompts can contain regulated data.
Knowledge bases and workflow nodes
Your model appears in the LLM, embedding and rerank selectors of workflows and knowledge bases, so each model type you expose must behave correctly in every node that uses it. An embedding model that returns vectors of an unexpected dimension will break indexing, and a model that does not support a requested feature will fail mid-workflow. Test each node type the business plans to use before publishing the plugin to users.
How DeployOpen scopes and delivers a model plugin
DeployOpen plans, builds, tests and documents custom Dify model provider plugins inside your own infrastructure, so your internal model gateway or fine-tuned model appears in Dify with the credentials and model types you define. Engagements are fixed-scope, with scope and deliverables agreed in writing before any build starts.
How we scope
- Confirm which model types the internal gateway or fine-tuned model will expose inside Dify
- Map the credential schema, including which keys, endpoints and tenant values each workspace needs
- Record the decision that tools belong in a separate tool plugin, since model plugins cannot bundle them
- Agree the Dify version, plugin daemon setup and target environments in the written scope
How we build
- Implement the provider manifest, model schemas and credential validation against your gateway
- Route model calls through your approved endpoints so traffic follows your network and egress rules
- Test each exposed model type in a staging Dify instance before any production package is shared
- Package the plugin and check signing and installation steps match your distribution policy
How we hand over
- Deliver the plugin source, packaged build and version history in your own repository
- Document the credential schema, supported model types, configuration keys and upgrade notes
- Walk your platform team through installing, rotating credentials and rolling back a plugin version
- Provide test cases so your engineers can rerun validation after each Dify or gateway change
How we support
- Review plugin compatibility when Dify minor releases arrive, which happen roughly monthly
- Update the plugin when your gateway endpoints, model catalog or credential rules change
- Ongoing support, upgrades and continuous testing are available as a retainer after handover
- Confirm support terms, response expectations and release cadence in the written agreement
