How Dify connects to Vertex AI models for enterprise use
A regulated team that wants Gemini or partner models inside its existing Google Cloud contract, while keeping Dify in its own infrastructure, should use the langgenius Vertex AI model provider plugin from the Dify Marketplace, authenticated with a Google Cloud service account. That plugin is the supported connector, and this page does not describe a custom integration. The plugin reads the Vertex AI project and location settings you enter, sends model calls to Google Cloud on your behalf, and returns results to the Dify workflow or app that requested them. The Dify Marketplace listing for Vertex AI is the starting point for confirming the plugin's current version and its configuration fields before you install it.
Treat the service account as the main control point. Create a dedicated account for each Dify environment, grant it only the Vertex AI roles needed for the models you have approved, and store its JSON key in the Dify model provider configuration rather than in application code or workflow variables. Confirm with Google Cloud documentation which roles cover prediction calls for the partner models you plan to enable, because role names and model availability change. Also confirm which Vertex AI locations host each model, since the location you choose determines where prompts and completions are processed. Choosing Vertex over the direct Gemini API is usually a contract and governance decision: it lets you apply existing Google Cloud agreements, billing, IAM audit logging and regional settings to model traffic. Ask your Google Cloud account team which terms apply to your organization before you make that choice.
What you need before connecting Dify to Vertex AI
Connecting Dify to Vertex AI touches your cloud account, your identity setup, your self-hosted runtime and your finance approvals. Work through these items before anyone edits the model provider settings, so the connection is built on a project, a credential and a network path your team already controls.
Google Cloud project with Vertex AI enabled
Dify calls Vertex AI inside a specific Google Cloud project, so that project must exist, belong to your organization and have the Vertex AI API turned on. Decide which project will hold model traffic and whether it is separate from development and production workloads. Confirm the current API enablement steps in Google Cloud documentation, because console menus and service names change.
Service account with a model-call role
Create a dedicated service account for Dify instead of reusing a person's identity or a broad project owner account. Grant only the Vertex AI role that permits model calls, and confirm the current role name in Google Cloud documentation before assigning it. Plan a key rotation owner, because the JSON credential Dify stores becomes a secret that must be protected like any other.
Dify version and self-hosting requirements
Confirm which Dify release you run and that the host meets the Docker Compose baseline in the self-hosting docs: at least 2 CPU cores, 4 GB of RAM and Docker Compose 2.24.0 or later. Minor releases arrive roughly monthly, so record the version you tested against. The Vertex AI provider is installed as a plugin, so check that your plugin daemon is running before you start.
Outbound access through the SSRF proxy
Dify routes outbound requests through an SSRF proxy that blocks internal and private IP ranges. Your Vertex AI endpoints are public Google addresses, so the proxy and any corporate egress firewall must allow HTTPS traffic to them. Test the path from the Dify API container, and confirm with your network team whether Private Google Access or a proxy allowlist is required in your design.
Billing account approval
Model usage is billed to the Google Cloud billing account linked to the project, so someone with finance authority must approve that link and the budget limits. Ask who owns the billing account, whether alerts are configured, and how usage will be attributed to Dify workspaces. Cost depends on the models, token volume and data sent, so request a usage estimate from the model owner before go-live.
Configure the Vertex AI provider in Dify step by step
This sequence connects a self-hosted Dify workspace to Gemini and partner models in Vertex AI, from plugin installation through model selection and workspace defaults. Each step ends with a check so your team can confirm the connection works before any application depends on it.
Confirm the Dify deployment and workspace
Before adding a provider, confirm that the Dify instance is running the version your team intends to keep, and that you are working in the workspace that will own the Vertex AI connection. Your platform owner should check the release in use against the Dify releases page and note any pending upgrade, because the plugin step depends on the plugin architecture in the running version. You are done when you can sign in as an administrator, open the model provider settings, and see the Vertex AI option listed among the available providers.
Install the langgenius Vertex AI plugin
In the Dify Marketplace, locate the Vertex AI model provider plugin published under the langgenius account for Google Cloud Platform, and install it into the workspace. Confirm the publisher name and plugin page before installing, since plugin trust is a separate decision your security team should review. The check is that the plugin appears under installed plugins with its version recorded, and the Vertex AI provider card becomes available in the model provider settings.
Create a dedicated service account key in Google Cloud
In the Google Cloud project that will bill and host the model calls, create a service account dedicated to Dify and grant it only the Vertex AI permissions needed to call the models you plan to enable. Generate a JSON key for that account. Your cloud team owns this step, and the output is a key file plus a record of the granted role and the project it belongs to. Store the key according to your secrets policy from the start, since the file grants the same access as the account behind it.
Enter the project ID and location
In the Vertex AI provider form, paste the service account JSON content as directed by the plugin form, then enter the Google Cloud project ID and the Vertex AI location (region) where your models are served. Use the location your data residency review approved, because model availability and the region that processes requests both depend on this value. The check is that the form saves without a validation error. If validation fails, confirm the project ID is the one the service account was granted in and that the location supports the models you want.
Select the Gemini and partner models to enable
Open the provider's model list and enable only the Gemini and partner models that your use cases require, separated by type where the form allows (chat, embedding, rerank if offered). Keep the list short so that application builders choose from approved models only. Confirm each enabled model's exact name against the Vertex AI model catalog in your project, since availability varies by region and the names in the Dify list must match what your project can call. The check is that each enabled model shows as active in the provider list.
Run a test chat call and read the console log
In a non-production workspace or test app, send a short prompt through each enabled model and confirm a response returns. Then open the Dify console logs for that test run and check that the request shows the expected model, the provider, and a successful status, with no unexpected errors from the credential or region. Record the test prompt and result in your change log. Do not paste the service account key or sensitive prompts into tickets or chat during testing.
Set workspace default models
Once tests pass, set the workspace default chat, embedding and rerank models to Vertex AI entries only if your architecture calls for it, and confirm that applications inherit those defaults as expected. Defaults decide where data leaves your network for apps that do not name a model explicitly, so review them with your data owner. The check is that a newly created test app picks up the intended default model and that the default setting is recorded in your configuration documentation.
Document the key handling and rotation plan
Record where the service account key is stored, who can read or replace it, how it is rotated, and how a compromise would be handled, following your organization's secrets policy. Confirm that the key is not exported with app DSL files, since Dify's export notes state that third-party API keys are excluded, which means each target instance needs its own credential setup. The check is a short runbook entry approved by your security owner and linked from the workspace's operations documentation.
When Vertex AI is preferred over the direct Gemini API
Choose Vertex AI over the direct Gemini API when the main reasons are contractual and governance-related rather than model-related. The usual drivers are that model usage is billed on the same Google Cloud agreement and invoice as the rest of your cloud estate, that access is governed by the same IAM roles and audit trails you already run, and that you can pick the region where requests are processed. Those are the reasons to evaluate, but they do not decide the question by themselves. The reader must compare the actual terms, data handling commitments and regional options in their own Google agreements, because the applicable terms depend on the contract, the product and the account configuration. DeployOpen does not assert which terms apply to your organization.
In the Dify setup, the choice shows up in the Vertex AI model provider plugin, which is configured with a Google Cloud project, a location and a service account rather than a single API key. Confirm with your Google account team which data handling terms, retention settings and regional processing commitments cover each Gemini and partner model you plan to enable, and whether those commitments are the same for the Vertex AI endpoint and the direct Gemini API. Also confirm which models are offered in your chosen location, since availability can differ by region and by model.
Before deciding, write down the requirements the decision must satisfy: who approves model spend, which region your data is allowed to be processed in, how service account keys are issued and rotated, and where request logs must be retained. If Vertex AI meets those requirements under your agreement, the Dify side is largely a matter of project, location and credential handling. If it does not, the direct Gemini API may still be the right route, but the same checks apply to it, and the Dify configuration will differ. Record the decision and the evidence behind it so the choice can be revisited when Google changes its terms or model availability.
Mapping Dify model settings to Vertex AI project, location and model fields
This table matches each Dify Vertex AI provider field to the Google Cloud value it represents and to the team that should own it. Read it before filling in the provider form, then confirm each model identifier and regional availability against current Google documentation before saving.
| Dify field | Vertex AI counterpart | Who owns the value |
|---|---|---|
| Service account credential | JSON key or identity for a service account with Vertex AI access, used to call the endpoint | Platform or security team, issued and rotated under your key policy |
| Project ID | The Google Cloud project that holds the Vertex AI API enablement, billing and quotas | Cloud platform team, per environment such as development and production |
| Location | The Vertex AI region where the model is served, which sets the endpoint and data processing location | Architecture and data governance, chosen against residency requirements |
| Model name | The published Gemini or partner model identifier available in that project and location | Application owner, confirmed against current Google model lists before saving |
| Model type | Chat, embedding or rerank role that Dify uses to route the call | Application owner, matched to the model's supported modality |
| Model availability check | Regional and model availability can change, so the identifier must exist in the chosen location | Platform team, rechecked before each upgrade or region change |
Securing the service account, credentials and outbound traffic
A Vertex AI connection puts a Google Cloud service account credential into Dify, and that credential can call paid model endpoints from inside your network. The cards below cover how to scope that identity, separate environments, store and rotate its key, limit who can change providers, and keep Dify's outbound requests from reaching internal addresses.
Least-privilege IAM for the service account
Grant the Dify service account only the Vertex AI permissions needed to call the models your apps use, such as prediction access in the chosen project, and nothing for administration, billing or other projects. Review the granted roles against the model list enabled in Dify, since broad project roles give any leaked key access to every Vertex resource in that project. Confirm the exact role names with Google Cloud's current IAM documentation before you write the policy.
Separate projects and service accounts per environment
Use one Google Cloud project and one service account for each Dify environment, such as development, staging and production, so a test key cannot spend production quota or read production data. Keep the Vertex location and project ID in each environment's configuration so a mistaken copy is visible in review. Ask your cloud team how project-level quotas and billing labels will separate the cost of each environment.
Key storage and rotation
Service account JSON keys are long-lived credentials, so decide whether your policy allows downloaded keys at all or requires a keyless alternative for workloads that can use one. If keys are used, record who created each key, set a rotation schedule, and replace the key in Dify without downtime by adding the new credential before revoking the old one. Check your organization's key creation policies before the first key is issued.
Protecting stored credentials with SECRET_KEY
Dify stores provider credentials in its database, and the environment reference says SECRET_KEY encrypts OAuth credentials. Set a strong, unique SECRET_KEY before first use, because changing it after deployment logs out all users. Workspace encryption keys can be stored locally or in Azure Key Vault, so confirm where your workspace keys live and who can read the database and backups that hold the encrypted Vertex credential.
Restricting who manages model providers
Any workspace member who can edit model providers can replace the Vertex credential and redirect traffic to another project. Limit provider management to a small named group, and keep app builders in roles that can use models without seeing the credential. For larger teams, Dify Enterprise offers SSO and central credential assignment, which can keep the key out of individual workspaces. Confirm the role model available in your edition before you finalize the access matrix.
SSRF proxy for outbound requests
Dify's SSRF_PROXY settings route outbound requests through a proxy that blocks internal and private IP ranges, which limits what a prompt-driven tool call or a misconfigured endpoint can reach. Confirm that the Vertex endpoints are allowed through the proxy, and that your firewall only permits egress to Google Cloud addresses from the Dify host. Test the block by confirming that a request to an internal address fails from inside a Dify tool or workflow.
Common Vertex AI connection failures in Dify and how to fix them
This table lists the failures teams most often hit when Dify calls Vertex AI, with the likely cause and the check to run for each. Work down the rows in order, since a wrong key or project often hides the later problems. Confirm exact error text in your own Dify logs and in Google Cloud audit records, because the wording varies by version and provider plugin.
| Failure | Likely cause | Check to run |
|---|---|---|
| Authentication rejected | The service account JSON key is malformed, pasted incompletely, disabled, or belongs to a different project than the one configured in Dify. | Confirm the key is enabled in the IAM console, re-paste the full JSON without edits, and check the service account's project matches the project ID in Dify. |
| Wrong project or location | The project ID or region in the Vertex AI provider settings does not match where the model is deployed or enabled. | Compare the project ID and location in Dify with the Vertex AI console, and test the same model in that location from a clean client. |
| Vertex AI API not enabled | The Vertex AI API is not turned on in the target Google Cloud project, so calls fail even with valid credentials. | In the project's API library, confirm Vertex AI is enabled, and check that billing is active on that project. |
| Model not available in the selected location | The Gemini or partner model you chose is not offered in the configured region, or the model name differs from the one in the Vertex AI model garden. | Check the model's regional availability in Vertex AI documentation, then match the exact model identifier in Dify to the one you tested. |
| Quota or permission denied | The service account lacks the Vertex AI role needed for prediction calls, or the project's request quota for that model is exhausted or was never raised. | Review the service account's roles in IAM, then check quota usage for the model in the project's quotas page before raising limits. |
| Outbound request blocked | The SSRF proxy or network egress rules stop Dify's worker from reaching Google API endpoints, often because the proxy allow list or firewall was tightened for other traffic. | Test egress from the api and worker containers to the Google endpoint, review SSRF_PROXY settings and proxy logs, and confirm the firewall permits the required outbound destinations. |
| Credentials visible after export | Teams sometimes assume the key moved with an app, but DSL exports do not include third-party API keys, so the imported app fails authentication. | Re-enter the Vertex AI credential in each target workspace and confirm the imported app points at the intended project and location. |
Questions teams ask before connecting Dify to Vertex AI
How much effort does connecting Dify to Vertex AI take?
The basic connection is small: install the Vertex AI provider plugin from the Dify Marketplace, enter a service account with its project and location, then test a model call. Most of the effort sits around that step. Cost rises with IAM role design, separate Google Cloud projects for development and production, secret storage, the network path from Dify to Google, and decisions about which workspaces may use which models. Ask for a quote that lists each of these as its own line so you can see what is one-time setup and what your team keeps running.
Can our own team connect Dify to Vertex AI without a partner?
A team that already administers Dify and manages Google Cloud IAM can make the basic connection alone, using the Vertex AI plugin in the Dify Marketplace. The risks are in the details that are easy to get wrong: a service account with broader roles than model calls need, a key copied into shared documents, or one project serving both test and production traffic. Teams without an engineer who reviews Google Cloud access regularly often bring in a fixed-scope review for those decisions before go-live.
What do we need from our Google Cloud and Dify administrators?
Four things, before work starts:
- Google Cloud: a project with billing linked and the Vertex AI API enabled, a service account limited to the roles your model calls require, and the project ID and location for each model you plan to use.
- Dify: a workspace administrator who can install plugins and enter provider credentials, plus a named owner who decides which workspaces can use the provider.
- Security: an agreed place to store the service account key, a rotation owner and a documented removal step.
- Both teams: a shared list of environments, so development and production never share a credential.
How long does a Vertex AI connection take when approvals are involved?
The configuration itself usually moves quickly once access exists, so the elapsed time is mostly waiting on approvals. Typical waits include creating or assigning a Google Cloud project, linking billing, approving the service account and its roles, passing a security review of where credentials live, and scheduling a production change window. This page does not give a duration. Ask any implementation partner to map each step to the approver who owns it, and to state which steps can start in parallel so the schedule reflects your real approval path.
Should we use Vertex AI or the direct Gemini API for enterprise terms?
Vertex AI tends to fit teams whose contracts, data location commitments, IAM and audit logging already run through Google Cloud. The direct Gemini API can fit teams whose agreement covers that offering and who do not need cloud-level access control around model calls. The deciding factors are the contract you hold with Google, the regions and data handling terms that apply to each path, and where your security team wants access and logs to live. This page does not state Google's contract terms, so confirm them in writing with your Google account team before you choose.
Who pays for model usage when Dify calls Gemini through Vertex AI?
Google bills model usage to your Google Cloud account, so that charge recurs and grows with traffic. Dify's Community edition is free and limited to a single workspace, while the self-hosted Enterprise edition is custom priced, according to the Dify pricing page. DeployOpen's fees come only from a written fixed scope. Usage cost rises with call volume, model choice, prompt and output length, retrieval size in knowledge bases and the number of environments. Ask for a quote that separates one-time integration work, your ongoing Google usage and any retainer for support.
What changes for the Vertex AI connection after a Dify upgrade?
Dify minor releases arrive roughly monthly. Dify 1.16.0 was released on July 17, 2026, and 1.17.0 on August 25, 2026, according to the Dify releases page. Before each upgrade, take a backup, read the upgrade guide for the target version, and check the current .env.example for new variables. Afterward, retest a Vertex AI model call in each workspace, because plugin and provider behavior can change between versions. Keep SECRET_KEY unchanged during upgrades, since changing it logs out all users.
Keeping the Vertex AI connection healthy after go-live
The Vertex AI connection needs an owner after go-live, because three things change under it: Dify itself, the service account credentials, and Google's model catalog. Plan a recurring check for each, starting with monitoring model calls and failures in the Dify console and in your own logs. Track failed invocations by model name, workspace and app, since an authentication error, a quota limit or a disabled model usually shows up first as a spike in failed runs rather than as a single alert. Confirm with your Google Cloud administrator where Vertex AI request logs and quota metrics are visible, and who receives alerts when calls start failing.
Service account keys need a rotation procedure that is rehearsed before it is needed. Create the replacement key in the same project, update the credential in Dify for each environment, run a test call against each enabled model, and only then disable the old key in Google Cloud. Confirm your organization's key policy, including whether key creation is restricted, because some organizations prefer short-lived credentials or keyless methods where the environment supports them. Record which service account, which project and which location each Dify workspace uses, so the rotation is traceable during an audit or an incident.
Re-test after each Dify minor release. The Dify releases page shows minor versions arriving roughly monthly, and upgrade guidance tells operators to back up before each upgrade and to check .env.example for new variables. After every upgrade, run a fixed test set against each Vertex AI model you use: a chat call, a call that uses your planned parameters, and an embedding call if you use one for knowledge bases. Compare results with a stored baseline. Review the enabled model list as Google changes its catalog, retiring model versions and adding new ones, and remove models no app uses so that the list stays short and the next audit stays simple.
