Flowable external worker tasks for polyglot services

How Flowable external worker tasks run process work in your own services

A process model often needs a step whose logic lives in a service your Java team does not own, or in a language the engine does not run. An External Worker task solves this by creating a job inside Flowable and leaving the work to a worker that runs anywhere. Workers written in any language acquire those jobs over the Java API or the REST API, execute the logic, and report back to the engine. In the model, the task is a service task with flowable:type set to external-worker and a topic, which can be an expression, that workers query. It behaves much like an async service task, except that the code runs outside Flowable.

Each External Worker task therefore has three parts you must design: the topic name, the worker service that polls for jobs under that topic, and the variables it reads and writes. Agree topic names with the teams that own the workers and keep them stable, because a model that publishes a topic no worker listens for will wait indefinitely. Confirm with your Flowable version how the job acquisition call is authenticated and which REST endpoints your worker will use, since the worker is a separate client that the engine's own security settings must allow. Decide early which variables go in the job payload and which the worker returns, so both sides agree on names and types before any model is deployed.

The engine keeps the job's state, lock and retry information, so the worker's job is to acquire, execute and then complete or fail the job explicitly. Plan for workers that crash after acquiring a job, since the lock must expire before another worker can pick it up, and check the lock behavior in the Flowable advanced documentation for your version. Ask the engineering team how failures are reported back to the process, how many retries are allowed, and where failed jobs appear for operators. Test these paths deliberately, because a worker that works on the happy path can still leave process instances stuck in production.

When external workers beat Java delegates or HTTP tasks

Each execution option moves a different set of costs onto your team. The decision usually turns on who owns the logic, how often it changes, and how the call behaves under failure. The cards below compare external workers with Java delegates and the HTTP task so you can see what each choice makes you operate.

Logic owned by another team or language

If the business logic lives in a Python, Node.js or Go service owned by a different team, an external worker lets that team write and run the code in its own language. The process model only references a topic, so the Java engine never loads that code. The cost moves to you: someone must run the worker fleet, handle its authentication to the engine REST API, and agree on the variables each topic expects. Check who will be on call for those workers before you commit.

Code that deploys independently of the engine

A Java delegate is compiled into the engine application, so each logic change means rebuilding and redeploying the process application. An external worker runs as its own deployable, and its release cadence can differ from the process definitions. That separation helps when fraud rules or enrichment code change weekly. The trade-off is version coordination: a worker expecting an old variable name can fail jobs after a model change, so you need a contract for topics and payloads.

Long-running or retry-heavy calls

External worker jobs are acquired and locked by a worker, and the worker reports completion or failure back to the engine. A slow partner call or a batch step can hold a job for a long time without blocking engine threads, and a failed job can be retried under engine control. A Java delegate running the same call inside the async executor ties up engine resources, and a plain HTTP task gives you fewer hooks for custom retry logic. Confirm the lock and retry settings with your engineers before you size the worker pool.

Short calls where a delegate is simpler

For a quick in-process calculation, a validation rule or a database lookup that your Java team already maintains, a Java delegate is usually the least work. It needs no topic, no job polling, no worker process and no REST credentials for a second client. Unit tests run inside the same build as the process application. Choosing an external worker here adds a deployment and a failure mode without adding a capability you need.

Stable REST calls where the HTTP task fits

When a process needs a single request to a well-defined REST endpoint, the HTTP task keeps the call inside the BPMN model with request and response mapping. You avoid running any worker code at all. It becomes the weaker choice when the response needs multi-step handling, custom parsing, secret rotation logic or a language-specific SDK. Ask the team that owns the endpoint whether its client library is needed, and check how the HTTP task handles timeouts and errors before relying on it for critical steps.

Operating a second service

Every external worker adds a long-running process with its own identity, configuration, scaling rules, logs and upgrade path. Java delegates and HTTP tasks run where the engine or the target system already runs, so they add fewer moving parts. Before choosing workers, ask who will patch the worker runtime, monitor job backlog and rotate its credentials. If that team does not exist yet, the operational cost may outweigh the benefit of language choice.

Prerequisites and configuration sequence

This sequence takes a team from confirmed prerequisites to a verified external worker loop, in which a Flowable process creates jobs on a named topic and a worker outside the Java engine acquires, executes and reports each one. Each step closes with a check, so problems surface before the worker reaches production.

Confirm the engine version and API surface
Before any design work, the platform owner confirms which Flowable engine version is in use and whether the team will reach external worker jobs through the Java API, the REST application, or both. Workers written in other languages normally depend on the REST endpoints, so the REST application must be deployed and reachable from the worker network. Confirm the exact version against the release notes and the Flowable docs for that version, since external worker behavior is documented per release. Check: a written statement names the engine version, the API the worker will use and the network path from worker to engine.
Confirm the database and async executor setup
External worker jobs are stored in the engine database, so the platform owner confirms which supported database type the engine runs on and that the database is shared by every node that will serve jobs. Confirm the database version against the Flowable configuration docs, including any minimum version requirements noted there. Confirm that the async executor is enabled on the nodes that run the engine, because lock owners and lock expiration times in the database govern which node holds a job. Check: a test connection from the engine to the database succeeds on every node, and the async executor configuration is recorded in the environment's configuration baseline.
Create the worker account and authentication
Decide which identity the worker uses to call the REST API, create a dedicated technical account with only the privileges it needs to acquire, complete and fail external worker jobs, and store its credentials in the organization's secrets store rather than in worker source code. Confirm with the security team how the worker will authenticate, since the current REST setup may use basic authentication, and agree on a plan if the organization requires tokens. Check: the account can call the external worker job endpoints in a non-production environment and is refused on endpoints outside its role.
Define the topic name and its ownership
Agree on the topic name that the External Worker task will publish and the worker will query. A topic can be a fixed string or an expression, so the team decides whether one topic serves one capability or whether expressions route jobs by process variables. Record who owns each topic, which workers subscribe to it, and what the worker does when no worker is listening. Check: a topic register lists each topic name, its owning team, its subscribing workers and the expected job payload, and the names follow one agreed convention.
Model the External Worker task in BPMN
In the process model, add a service task, set the flowable:type attribute to external-worker, and set the topic to the name agreed in the previous step. Map the process variables the worker needs as inputs, and decide which values the worker will return as outputs, since the worker writes results back to the process instance on completion. Keep the model free of Java delegate references for this task so the logic stays outside the engine. Check: the model validates in the modeler or the deployment step, and a review confirms the topic, input variables and expected output variables match the register.
Deploy the definition and start a test instance
Deploy the process definition to a non-production engine using the deployment pipeline the team already uses, then start one test instance with sample variables. The engine creates an external worker job when execution reaches the task, and that job remains available until a worker acquires it. Confirm the job appears in the engine's job data for that instance before any worker code is written. Check: the deployment shows the new definition version, and the test instance is waiting at the External Worker task with a job queued on the expected topic.
Acquire, lock and execute jobs in the worker
Build the worker to query the REST API for jobs on its topic, acquire them with a lock, and run its business logic against the job's variables. The worker holds the lock for the duration of its work, so set a lock duration that exceeds the worst-case execution time of the job, and confirm the lock expiration behavior with the Flowable advanced documentation before choosing a value. Check: a single worker instance acquires the test job, the job shows as locked to that worker, and a second worker instance does not receive the same job while the lock is active.
Complete or fail jobs and verify the outcome
On success, the worker completes the job and returns its output variables, and the process continues past the task. On a recoverable error, the worker reports failure so the engine can apply its retry handling, and on a permanent error the team routes the instance to an error path it has modeled. Verify both paths with the test instance, then check the process history for the completed and failed events. Record the verification in the test evidence so the sequence can be repeated before each release. Check: the test instance advances after completion with the expected output variables, a forced failure produces the expected retry or error behavior, and the history reflects both outcomes.

How topics, job locks and retries shape worker behavior

The topic is the routing key between the process and the worker pool. An External Worker task is a service task with flowable:type set to external-worker, and its topic can be an expression, so a process variable such as a region or product code can select which pool receives the job. Workers query for jobs by topic, which makes the topic name a contract between the modelers who write the BPMN and the teams who run the workers. Name topics for a capability that will still exist after the next reorganization, agree the list before build, and keep a register of which pool listens to each one. A topic with no listening worker leaves its jobs queued without an obvious error, so a check that every deployed topic has a consumer belongs in the release process. Confirm the exact expression syntax in the BPMN constructs reference.

Acquisition is where a job changes hands. When a worker acquires a job, the engine records a lock owner and a lock expiration time for it in the database, so other workers and other engine nodes do not receive the same job during the lock period. Completing a job tells the engine the work is done and lets the process instance move past the External Worker task, carrying forward any results the worker reports back. Failing a job reports that the work did not succeed, and what happens next depends on the retry handling your configuration applies. Confirm in the docs how many retries a failed job receives, how long the engine waits between attempts, and what operations sees while a job sits in a failed state. Those answers decide whether a short outage in a downstream service is absorbed by retries or becomes an incident for the on-call team.

Lock expiration is the recovery path for a worker that crashes mid-job. If a worker stops before it completes or fails the job, the lock eventually expires, and the async executor, which runs asynchronous jobs from a thread pool on each node, can pick the job up again. Recovery depends on the lock owner and expiration values stored in the database, so the lock duration is a design decision. Set it too short and a slow but healthy worker can lose its job to a second worker, and set it too long and a crashed job waits longer before recovery. Derive the duration from measured task times with headroom, make worker logic idempotent so a repeated run does not duplicate a payment or notification, and test the path by stopping a worker mid-job in a non-production environment. Confirm the lock setting name and its default in the advanced documentation before you change it.

Choosing the language and runtime for each worker

Each external worker is a separate service with its own language, credentials, scaling rules and location. The choices below determine whether a worker stays dependable as the number of pending jobs grows, and each one should be recorded per topic in the design.

Language and client library

Pick the language for each worker based on the team that owns the business logic and the libraries it already runs in production. A worker only has to acquire a job, run its code and report back over the REST API, so the key question is whether a maintained client and solid JSON handling exist for that language. Confirm an official or community client for your exact language and version before the design is final. The Python and Node.js client pages cover those specifics.

Worker authentication

Each worker calls the engine with its own credentials, so authentication is a per-worker decision. Choose a method your engine version supports, store the secret in your secrets manager, and plan rotation before go-live. Give each worker a dedicated engine user that can acquire and complete jobs only for its own topics. Change or remove default accounts before any worker connects, and confirm the authentication options available in your version.

Scaling with pending jobs

Scale workers by the pending jobs on each topic, not by the number of running process instances. Track queue depth and wait time per topic, then add workers where work falls behind. Several workers can poll one topic, and the engine stores lock owners and lock expiration times in the database, so a job held by a stopped worker becomes available again once its lock expires. Confirm the lock duration and polling behavior for your version in the Flowable advanced documentation.

Deployment location

Run each worker where the data it touches is allowed to go. A worker that calls a payment or patient system often belongs in the same network zone as that system, which may be separate from the engine's cluster. Consider containers, a dedicated Kubernetes namespace or a virtual machine, and confirm that the worker can reach the engine's REST endpoint through the firewall, proxy and certificate chain. Plan the network path, certificates and egress rules as part of the design.

Pinning the client to the engine release

The REST contract belongs to the engine release, so the worker client has to match it. Flowable 8.0.0 was released on February 27, 2026, and moves the engine to Spring Framework 7 and Spring Boot 4 with no backward compatibility for Spring Boot 3. Record the engine version and each client version together in your deployment inventory, and retest every worker during an engine upgrade. Review the release notes for changed REST resources before you upgrade.

Failure and retry behavior

Decide what a worker does when its work fails: complete the job with output variables, report a failure with a message, or stop and let the lock expire. Set retry counts and backoff per topic, and treat every job as one that may run twice, so handlers must be idempotent. Confirm how your engine version handles failed external worker jobs and where operators can see those failures to act on them.

Mapping topics, variables and identities between the model and workers

This table lists each point where the BPMN model and the external worker must agree, from the service task definition to the worker identity. Read each row as a question the model owner and the worker owner answer together, and record the answers in one shared contract before the first job is created.

Contract point	What the model owner sets	What the worker owner must do
Model element and type	A service task with flowable:type set to external-worker, so the engine creates jobs instead of calling Java code	Confirm the element is the one that creates the job for their service and that no delegate runs in parallel
Topic expression	A topic value or expression that workers query for jobs, agreed as a fixed name or a documented variable	Poll only the topics they serve and match spelling and case exactly to avoid jobs that nobody acquires
Input variables	Names, types and whether each process variable is sent to the worker with the job	Read only the variables named in the contract and reject payloads with missing or differently typed values
Output variables	Names and types of variables the worker returns on completion, and which ones the model reads in later gateways	Return every promised variable on completion and never rename them silently
Job identifier	Nothing beyond keeping the job in the process instance, which the engine assigns	Use the job identifier as the idempotency key so a retried call does not apply the same work twice
Worker identity	Which identity or group may acquire jobs for each topic, using engine users, groups or REST credentials	Authenticate with the agreed identity and keep credentials per worker, not shared across services
Success outcome	Expected next step after completion and any variables the next activity requires	Signal completion only after the external side effect has finished or is recorded as safe to repeat
Retryable failure outcome	Retry count and the behavior the model expects while retries remain	Report a failure with a message that explains the cause so operators can act on the failed job
Permanent failure outcome	Boundary error handling or an escalation path for failures that retries will not fix	Classify errors as permanent in the failure report so the model routes them to the handler instead of retrying

Running workers in shared, regulated or air-gapped environments

In a shared cluster, the Flowable engine and the external workers run on the same Kubernetes or virtual machine platform, often in the same namespace family or network segment. Workers call the engine's Java API or REST API over internal service addresses, so the main design question is isolation between teams rather than reachability. The worker pods are usually owned by application teams, while the engine belongs to the platform team, and both share the same cluster policies.

Service addressing: Workers reach the engine through an internal service name, and that name should resolve only inside the cluster or approved namespaces.
Network policy: Restrict ingress to the engine so only labeled worker workloads can call its REST endpoints.
Credentials: Give each worker group its own engine user with rights limited to the topics it handles, and store those secrets in the cluster's secret store.
Resource limits: Set CPU and memory limits on workers so a busy topic cannot starve the engine's async executor.
Shared database: Confirm which schema and connection pool each team may use, because workers that poll heavily add load to the same database.

Confirm with the platform team which namespaces and labels are allowed to call the engine, and with the security team which service accounts are permitted to hold engine credentials.

Securing the worker channel and the data it carries

External worker tasks put a network channel between the Flowable engine and code you may run in other zones, languages and teams. The cards below cover the settings that control who can take jobs, how traffic and credentials move, and what ends up in logs. Review them together, because a weak choice in one usually shows up as a problem in another.

Authenticate every REST call from workers

Workers that acquire and complete jobs over the Flowable REST API need an identity the engine accepts. Basic authentication with a shared account is the simplest option, but it gives every worker the same rights and makes it hard to trace which service completed a job. Decide whether each worker type gets its own user, and confirm which authentication methods your Flowable version and configuration accept before writing the worker client.

Limit each worker to its topics

A worker should only be able to acquire jobs for the topics it handles and only complete jobs it has locked. Check whether the identity and authorization setup in your deployment lets you restrict access by topic, process definition or group. If it does not, plan the restriction in the worker code and in your review process, and test that a worker for one topic cannot take jobs for another.

Encrypt traffic between zones

Job payloads and completion results move between the engine and workers over HTTP. When workers run in another network zone, cloud account or language runtime, terminate TLS at a point you control and verify certificates in the worker client rather than disabling checks to get a connection working. Confirm where your reverse proxy or ingress ends TLS and whether traffic inside the cluster is also encrypted.

Keep worker secrets out of models

Process variables and BPMN models are stored, versioned and often exported for review, so credentials for downstream systems do not belong in them. Give each worker its own secret store entry, rotate it on a schedule you set, and keep only references in process variables. Ask your security team to confirm where worker credentials are stored and who can read them.

Control payloads in logs

Job variables often contain customer, employee or patient data, and worker frameworks log requests by default. Decide which variables a worker may log, mask fields that identify a person, and set log retention to match your data protection rules. Check the engine's history and audit settings too, since completed job data can persist there longer than the worker logs.

Restrict network paths to the engine

Only the worker zones that need the job endpoints should reach the Flowable REST application, and only on the ports and paths they use. Use firewall rules or network policies that name the worker subnets or services, and block direct database access from workers. The Flowable identity service can align worker users with the rest of your identity setup, and the Keycloak topic covers token-based access as a next read.

Common external worker failures and how to fix them

This table lists the failures teams most often hit when external worker tasks run outside the Java engine, with the likely cause and the first fix to try for each. Read it as a triage order: confirm the symptom in the job list first, then check the cause column before changing configuration. Most of these come down to topic naming, lock timing or the identity used for REST calls.

Failure	Likely cause	First fix to try
Jobs never acquired by any worker	The topic on the task does not match the topic the worker queries, often because an expression resolved to a different value or the names differ in case or spacing	Compare the resolved topic on the task with the worker's topic string exactly, then log the topic the worker sends on each poll
Jobs stuck with no worker holding them	A worker acquired jobs and crashed or stopped without releasing them, so the lock stays active until it expires	Wait for the lock to expire so the job becomes available again, then fix the worker's shutdown and error handling so it releases or fails jobs it cannot finish
Completion rejected after lock expiry	The worker ran longer than the lock duration, so the job was released and may have been acquired by another worker	Shorten the work per job or split it, and confirm the lock duration with your Flowable version before tuning; make completion handlers idempotent
Same job failing and retrying repeatedly	The worker reports failures for errors that will not change on retry, such as invalid input or a missing downstream record	Report a failure with a clear message for permanent errors, route them to a boundary event or a review task, and reserve retries for transient errors
401 or 403 from the REST API	Worker uses basic credentials or a token that lacks the privileges to acquire or complete external worker jobs, or the account is disabled	Check the worker account in the identity store, grant the required Flowable privileges, and confirm the credentials the worker actually sends
Process variables missing or wrong type after completion	The variable names or JSON types in the completion request do not match what the model expects, or the output mapping is absent	Compare the variable names and types in the completion payload with the task's output mappings and expressions, then test with one job in a dev environment
Worker cannot reach the engine	Network path, base URL or TLS trust between the worker and the REST application is wrong, often after a deployment change	Confirm the base URL and port from the worker host, then check certificate trust and any proxy or ingress rules

Questions teams ask before building external workers

How much effort does a first external worker take?

The first worker is mostly about agreeing the contract between the process and the service: which topic it listens on, which variables go in and out, how it reports success and failure, and how it is deployed. The effort depends on how many external task types the process needs, how much the existing service logic has to be wrapped, and how many environments the worker must run in. Ask any partner for a written breakdown of those items, separating design, build, testing and handover, so you can see which parts scale with your number of topics and services.

What does the work need from our team?

Expect to provide four things: access to a test Flowable engine with permission to deploy process definitions, network rules that let the worker reach the engine's REST or Java API and the services it calls, a named owner for each worker service, and a sample of the business logic or API contracts it must implement. Your security team will also need to decide how workers authenticate, since the engine's default accounts should not be reused. Confirm these access paths early, because waiting on firewall changes or accounts is a common cause of delay.

Can our internal team build external workers without outside help?

Yes, if the team already runs Java or Spring services and can operate the Flowable engine. Workers are ordinary services in their own language that poll for jobs, lock them, and complete or fail them, so the main new skill is understanding the job lifecycle and retry behavior. Teams without BPMN or engine experience usually need more time on modeling and failure handling. A fixed-scope engagement can cover the first worker and a handover so your team can add the rest, which is useful when you want the pattern without carrying the learning curve alone.

When is it worth bringing in a partner?

A partner is worth considering when the process crosses several systems with strict security or audit needs, when the engine is used in production and failures would affect customers, or when your team lacks capacity to design the worker pattern and test it. Bring in outside help before the first build starts, so the topic naming, error handling and deployment pipeline are agreed in writing. DeployOpen plans, implements, hardens and documents these integrations inside your own infrastructure under a written scope.

How does the work change when we run many workers?

At scale, the questions shift from getting one worker running to managing many. You need clear topic ownership, consistent retry and failure rules, monitoring of waiting and failed jobs, and a deployment process for each worker language. The async executor coordinates nodes through lock owners and expiration times stored in the database, so workers and engine nodes must be sized and tuned together. Ask for a plan covering job volume, lock timing, dead-letter handling and who responds when a worker stops acquiring jobs.

What goes wrong most often with external workers?

The most common problems are workers that acquire jobs but never complete them, topics that do not match between model and code, and retries that repeat side effects such as payments or emails. Lock expiration can also cause a job to be picked up twice if a worker runs slower than expected. Engineers should check that every task is idempotent or guarded, that failures are reported with clear error details, and that timeouts are set to match the real service latency. Test these cases before go-live rather than in production.

Does the engine's security model cover worker authentication?

Not by itself. Workers call the engine over the Java or REST API, so the authentication method, the account they use and the scope of its privileges must be chosen deliberately. Flowable's documentation describes replacing identity management through its identity service interfaces, which affects who can acquire jobs. Ask your security team to review the account model, token or credential storage and audit logging for worker calls before the worker reaches a shared environment.

Where external workers fit alongside the other Flowable integration options

Choose the integration point by asking where the logic should run, who owns it, and how the engine should know when it has finished. External worker tasks suit work that must run in a service your team operates in another language, or under another runtime, while the process waits for a result. Work that is a short call to one REST endpoint and needs little logic can stay in the Http task, where the request, timeouts and response mapping live in the model. Work that belongs inside the same JVM as the engine, such as a rule or a small transformation, is usually better as a Java delegate. The external worker sits between these: the engine creates a job on a topic, a worker acquires it, and the worker reports completion or failure back over the Java or REST API.

Integration logic that calls a family of enterprise systems, or that needs Camel components for legacy protocols, points toward the Camel task. Inbound and outbound business events, such as a payment message that should start a case or correlate to a waiting instance, belong in the event registry with the broker-specific pages for Kafka, RabbitMQ or JMS. Those are event problems, not work-assignment problems, so an external worker is the wrong tool for them. Confirm with the messaging team which broker carries each event before the model is drawn, because the channel definition and the correlation parameters decide how the message reaches the right instance.

Once the worker shape is chosen, the language pages cover the client side: the Python page for Flowable's official client, and the Node.js and TypeScript page for the @flowable-oss/external-worker-client library. Every worker that calls the REST API needs an identity, so the Keycloak page is the next read when basic authentication is no longer acceptable. Decide that identity model before the first worker is deployed, since it affects which service account acquires jobs and which privileges it holds. Mixing several integration styles in one process is normal, but record which style owns each step so that operations staff can tell, from a failed job or a stuck instance, which component to inspect first.

How DeployOpen scopes and delivers Flowable external worker integrations

DeployOpen plans, builds and hands over Flowable external worker tasks inside your own infrastructure, starting from a written scope that names the topics, workers and services involved. The work covers the process side, the worker side and the operating model that keeps jobs flowing after handover.

How we scope

Inventory every process and case model that needs work outside the Java engine, with its topic names and expected job volumes.
Agree which worker languages, runtimes and owners will service each topic before any build starts.
Engagements are fixed-scope, with deliverables and timelines agreed in writing. Ongoing support is available as a retainer.

How we build

Configure external worker task definitions with agreed topics, expressions and job retry behavior in the process models.
Build worker services in the languages your teams use, acquiring and completing jobs through the Java or REST API.
Test success, failure and lock-expiry paths so failed jobs are reported with clear messages and recover as designed.

How we hand over

Document each topic, its payload contract, authentication method and the worker deployment steps your team will repeat.
Provide runbooks for stuck jobs, failed retries and worker scaling, written for the teams who will operate them.
Walk your engineers through the test suite and monitoring views so they can extend topics without DeployOpen.

How we support

Review worker health, job backlogs and error trends with your team on the cadence agreed in the retainer.
Plan upgrades of the Flowable engine and worker libraries as separate, scoped changes with their own test runs.
Confirm with your security team how worker credentials are stored, rotated and restricted to the topics each worker needs.