Flowable external workers in Python

How Python services run Flowable external worker jobs

If your Python team needs a BPMN service task to run Python code, the supported method is a Flowable external worker. The BPMN model uses a Flowable External Worker task that creates jobs on a named topic, and a Python service acquires those jobs, executes them and reports the result back to the engine. Flowable publishes an *External Worker Client for Python* on PyPI as flowable.external-worker-client, which is a library that connects your application code to the external worker functionality. The BPMN constructs documentation defines the model side: the task carries flowable:type set to external-worker and a topic that workers query for jobs.

The engine does not call your Python code directly. Instead, the process waits at the task while a job sits in the database, and a worker pulls that job over the Java API or the REST API. The worker runs its logic, then completes the job or reports a failure, and the process moves on. This pull model means the Python service can run in its own container, scale on its own schedule and be deployed without restarting the Flowable engine. Confirm with your Flowable version which REST endpoints your engine exposes, since the REST application and its deployment options are a separate decision from the engine itself.

Before you build, decide three things with your Flowable and platform owners. First, choose the topic names and the variables each job passes in and out, because these become a contract between the process models and the Python code. Second, confirm how the worker authenticates to the REST API and who issues those credentials. Third, agree how many Python workers will poll each topic and how long a job may stay locked before another worker can take it. The client repository on GitHub, linked from the topic sources, is the next place to check the current client API and its supported Python versions.

What a Python worker environment needs before the first job

A Python external worker is only useful once four things are in place: a Flowable engine that answers over REST, a Python runtime the official client supports, an identity allowed to work jobs, and a shared list of topic names. Confirm each one with the team that owns it before writing any worker code.

A reachable Flowable REST endpoint

The worker acquires and completes jobs through the Flowable REST API, so the engine must expose that API to the worker's network. Confirm which Flowable deployment serves it (the REST application or a Spring Boot starter with REST enabled) and the base URL the worker will call. Check that the engine version in use matches the REST paths your client version expects, and test a simple read request from the worker host before anything else.

A supported Python runtime

The Flowable External Worker Client for Python is published on PyPI as flowable.external-worker-client. Before you pin a version, read the Python version requirement in the PyPI metadata and the project repository, because the client's supported interpreter range decides which base images and operating system packages you can use. Record the exact version in your build pipeline so that worker deployments stay reproducible.

A service account with job rights

The worker needs an identity that can acquire jobs for its topics and report completion or failure. Ask the Flowable administrator to create a dedicated user, grant only the privileges for external worker job acquisition and completion, and avoid reusing a people account or the default administrator. Decide whether the worker uses basic authentication or a token, and confirm how the credential will be stored and rotated in your secrets system.

Network paths from worker to engine

Each worker must reach the Flowable REST endpoint over the network path your security zones allow, often through a proxy, load balancer or service mesh. Confirm the firewall rules, DNS names and TLS certificate trust for the worker's runtime, and whether workers in separate clusters or accounts can reach the engine without exposing the database. Test from the same subnet or pod where the worker will run.

Topic names agreed with modelers

External worker tasks in BPMN models carry a topic, and a worker only picks up jobs whose topic it subscribes to. Agree on topic names with the process modelers before development starts, using a stable naming convention, and record which topics each worker group serves. A topic that is renamed in a model after deployment can leave jobs waiting with no worker to take them, so confirm how model versions and worker releases will be coordinated.

Configuring the Python worker from install to first completed job

This sequence takes a team from an empty Python environment to one external worker job acquired and completed against a Flowable engine, with a check at each step so problems surface before the worker goes into a service. It proves the topic, the REST connection and the variable round trip on a test instance you control.

Install the Flowable Python client from PyPI
A developer installs Flowable's External Worker Client for Python from PyPI, under the package name flowable.external-worker-client, into a virtual environment that belongs to the worker service. Pin the version in your dependency file so that upgrades are a deliberate change. The step is done when the package imports cleanly in a fresh interpreter and the version reported by your package manager matches the one recorded in the lock file. Confirm the installed client version against the client repository before you build on it, because the API surface may change between releases.
Set the topic and REST base URL in configuration
Set the Flowable REST base URL, the topic name the worker will serve, and the credentials it will use in configuration outside the code, such as environment variables or a secrets store. Use a topic name that matches what your BPMN models will declare, and keep it unique within the engine so that unrelated workers do not compete for the same jobs. Ask your Flowable administrator which REST path your deployment exposes and which account has the privileges to acquire and complete external worker jobs. The step is done when the configuration loads without errors and the worker process reads the values from the environment you expect.
Model the external worker task in BPMN
In the process model, add a service task that Flowable treats as an external worker. Set the task's type to external-worker, using the flowable:type attribute, and set the topic attribute to the same value the Python worker is configured to serve. Give the task a clear name and document the input variables it expects, since the Python code will read them. Deploy the process definition to a test environment and open the deployed model to confirm that the type and topic values were stored as written. Check the Flowable BPMN constructs documentation if your version handles the attributes differently.
Start a test process instance
Start one instance of the deployed process on the test engine, passing the input variables your worker expects through the REST API or the Java API. Record the process instance ID returned by the start call. When the process reaches the external worker task, the engine creates an external worker job for that topic and holds the instance at that point. The step is done when the instance is active and its current activity is the external worker task, which you can confirm with a process instance query on the REST API.
Confirm the job exists before running the worker
Before starting the Python worker, query the engine for external worker jobs on your topic through the REST API. The query should return one job linked to the process instance ID you recorded, with its topic matching your configuration and its state showing it is waiting to be acquired. This check separates model problems from worker problems. If no job appears, the task type or topic is wrong, or the process did not reach the task, so return to the previous steps before going further.
Run the worker to acquire and complete one job
Start the Python worker with a handler registered for your topic. The handler reads the input variables from the acquired job, performs a simple operation such as computing a value or writing a log line, and returns output variables that the worker reports back when it completes the job. Run the worker once, let it acquire the job, and then stop it. The step is done when the worker log shows the acquisition and completion calls succeeding with no error responses from the engine.
Verify the job and output variables in the engine
Check the outcome in the engine. Query the process instance again and confirm that it has moved past the external worker task. Read the output variables the worker set, using the variable endpoint for the process instance, and compare them with the values the handler returned. Confirm that the external worker job no longer appears in the open job list. The step is done when the instance has advanced, the output values match what the handler produced, and no job for that instance remains open. Keep the test instance and its job record until your team has reviewed them.

When a Python worker beats a Java delegate or HTTP task

The choice between a Python external worker, a Java delegate and an HTTP task is less about technical capability than about where the work runs and who owns it. A Java delegate executes inside the engine's transaction, so its database changes commit or roll back together with the process state. That makes it the natural fit for Java-only logic that must be consistent with the engine. An HTTP task suits a simple synchronous REST call whose response can be mapped straight into process variables. An external worker suits work that a Python team owns, work that depends on Python libraries such as data science or document tooling, or work that needs its own scaling and deployment cycle. Those are the conditions under which the extra moving parts of an external worker pay off.

The decision depends on three things you should settle before writing the model. First, the team: who writes, tests and on-call supports the code. Second, the failure handling you need: an external worker runs as a job that the worker acquires, completes or fails over REST, so retries, lock expiry and failed-job handling are separate concerns from a delegate that throws an exception inside the transaction. Third, operational ownership: a worker is a separate process with its own deployment, credentials and monitoring. Confirm with your Flowable and Python teams how a failed worker is detected and who restarts it. This page does not rank the three options for every case, and a process with mixed steps may reasonably use all of them.

Mapping job variables, topics and identities between Flowable and Python

This table compares the Flowable side of an external worker job with the Python side that acquires and completes it. Read each row as a point to settle with the Flowable team before the first worker goes live. The main takeaway is that topic names, variable types and the acquiring identity must match exactly on both sides, and the Python client's exact field names and serialization behavior should be confirmed against its repository.

Flowable item	Python worker item	What to confirm
External worker topic (flowable:type set to external-worker)	Topic string passed when acquiring jobs	Topic names match exactly, including case, and each topic has one owning worker group
Job identifier and lock fields	Job ID and lock handling used to acquire and complete the job	Which job fields the Python client exposes and how lock expiry is reported back
Process variables read by the worker	Input values the Python code reads from the acquired job	Which variables the BPMN model exposes to the external worker and their expected names
Process variables written on completion	Output values the worker returns when completing the job	Which variables are written back, and whether the engine overwrites values set elsewhere
Variable types (strings, numbers, dates, objects)	Python types after JSON parsing	Date and decimal handling, and how null values are represented in each direction
JSON serialization of complex values	Python dictionaries and lists built from the payload	Maximum payload size, nesting limits and how the client encodes non-JSON types
Acquiring user or service account	Credentials or token the Python service uses to call the REST API	A dedicated account with only the privileges needed for external worker jobs, and who owns its password or token
Completing user identity	Identity recorded when the worker reports completion or failure	Whether completion is attributed to the service account or a user, and how audit logs will show it
Authentication method	Basic authentication or token configured in the client	Which method the Flowable version in use supports and where secrets are stored

Securing REST access for external workers

A Python external worker holds credentials that can acquire and complete live process jobs, so its access to the Flowable REST API needs the same care as any other service account. The cards below cover the controls that connect the worker to the engine: how it authenticates, where secrets live, how traffic is protected, which service identity it uses and which topics it may touch. Confirm each control against the client repository and your Flowable version before you rely on it.

Confirm the authentication method

Before designing anything, check in the flowable-external-client-python repository whether the client accepts basic authentication, a bearer token, or both, and how it passes credentials to the REST calls. Flowable's external worker REST endpoints sit behind the same authentication as the rest of the REST application, so the worker's method must match what your engine accepts. Verify by running the client against a test engine and confirming that a request with missing or wrong credentials is rejected with an authorization error, not silently retried.

Use a dedicated least-privilege account

Create a separate engine user for each worker group instead of reusing an administrator or a human account. Grant it only the privileges needed to acquire and complete external worker jobs and nothing for deploying models or managing users. Check the grants in your identity setup, then run a test that tries an administrative REST call with the worker credentials and confirms it is denied.

Keep credentials out of code

Load the worker's username, password or token from a secrets manager or from environment variables injected at runtime, never from source files or container images. Confirm that application logs and exception traces from the client do not print the Authorization header or the secret value. A quick check is to force a failed acquisition, then search the log output for the credential string.

Terminate TLS in front of the REST endpoint

Send all worker traffic to the Flowable REST application over HTTPS, so credentials and job payloads never cross the network in clear text. Decide whether the worker reaches the engine directly or through an internal gateway, and validate the server certificate in the Python HTTP client rather than disabling verification. Test by pointing the worker at an endpoint with an untrusted certificate and confirming the connection fails.

Restrict acquisition by topic

Each external worker task carries a topic, and the worker asks for jobs by topic. Run one service account per topic group, or one per worker deployment, so a compromised Python service can only acquire jobs for the topics it owns. Confirm that a worker configured for one topic cannot acquire jobs for another by creating a job on a second topic and checking that the first worker does not receive it.

Rotate and audit worker credentials

Plan a rotation schedule for worker passwords or tokens and document who can issue them. Review the engine's history and audit records for acquisitions and completions made by each worker account, so unexpected activity is traceable to one service identity. Verify rotation by changing the secret in the store, restarting the worker and confirming it reconnects while the old credential is refused.

Why jobs stay locked, fail or never reach the worker

This table maps the symptoms teams see when Python workers and Flowable external worker jobs disagree, with the likely cause, the fix and the Flowable state to inspect. Read it from the symptom column down, starting with the topic and permission checks, since most stuck jobs trace back to a mismatch in those two places. Confirm exact behavior against the Flowable docs for your version before changing lock or retry settings.

Symptom	Likely cause	Fix	What to check in Flowable
Jobs sit in the queue and the Python worker never receives them	Topic in the BPMN model does not match the topic the worker polls for	Align the topic string exactly, including case and spacing, in the model and the worker	The flowable:type is external-worker and the topic value on the task matches the worker query
Worker gets an authorization error when acquiring jobs	The REST user lacks the privileges or the worker identity is not set up for external work	Grant the worker's user the external job privileges on the REST endpoint and retest with that account	The user's privileges in the Flowable REST identity setup and the response code returned on acquire
Jobs are acquired but the worker never completes them	Lock expires before the worker finishes, so another attempt or node takes over	Keep handlers short, report progress in smaller steps, and confirm the lock duration you configured	Lock expiry time on the external job and the time between acquire and complete in worker logs
Jobs show repeated failures and then stop retrying	Retries are exhausted after the worker reports failure several times	Fix the root cause, then reset retries on the job and resubmit; avoid failing jobs for transient errors without a retry plan	Retry count on the job, the failure message recorded on the job, and any failed job entries in management views
Completion fails with a variable serialization or type error	Result variables are not JSON-compatible with the types the process expects	Send only plain JSON types for variables, convert dates and decimals explicitly, and validate payloads before completing	The variable names and types in the process definition and the error text returned on the complete call
Requests to the REST endpoint fail with timeouts, TLS or certificate errors	Network route, proxy, or certificate trust on the Python host does not match the Flowable endpoint	Confirm the base URL and port, add the internal CA to the Python trust store, and check proxy settings for the worker process	TLS handshake errors in worker logs, the endpoint address in configuration, and whether the Flowable REST app is reachable from the worker's network
Worker authenticates but gets unexpected 401 responses after a period	Basic credentials or token expired, or rotated without updating the worker	Load credentials from a secret store, refresh tokens before expiry, and restart or reload the worker after rotation	Identity provider or REST user status, token lifetime, and the timestamp of the first 401 in worker logs

Questions buyers ask before building Python external workers

How much effort does a Python external worker build take?

The effort depends on how many topics the process models use, how much business logic each worker carries, and how much error handling the process needs. A worker that only reads a few variables, calls one internal API and completes the job is a small task. Effort grows with each additional topic, each set of input and output variables that must be mapped, each external system the worker calls, and each rule for what counts as a retryable failure versus a business error. No duration is given here because the scope has not been written yet. Ask any partner for a fixed scope listing topics, variables, error paths and deployment targets before comparing numbers.

What drives the cost of building Python workers for Flowable?

Cost follows the number of topics, the variable contracts between the BPMN model and the Python code, the depth of error handling, and the number of workers that must run in parallel. Each topic needs a defined input and output contract, a tested handler, and a decision on what happens when the handler fails. Authentication setup, secret handling and packaging for your runtime add further work. Recurring costs come from maintaining those contracts when process models change and from running the workers in your own environment. When requesting a quote, ask which items are one-time build work and which are ongoing support, and ask for the topic list and test approach in writing.

Can an internal team build Python external workers without outside help?

Yes, if the team has Python engineers who can call REST APIs, handle authentication and write automated tests, and someone who understands BPMN models and Flowable job lifecycles. The Python client is a library that connects application code to Flowable's external worker functionality, so the main new skills are the job acquisition loop, completion and failure calls, and operating long-running services. Teams without prior Flowable experience usually need time to learn how jobs, locks and retries interact with the process engine. A partner can help with the first topics, a review of the design, or a handover, which is a fixed-scope option agreed in writing before work starts.

What does the process team need to provide for external workers?

The process team owns the BPMN models and needs to define which service tasks use the external worker type, the topic names, and the variables each task reads and writes. They also need to decide which failures are business outcomes that the model should route, and which are technical failures that should be retried. Good inputs are a written list of topics with their variable contracts, example payloads, and the expected results for each path. Without those inputs, developers end up guessing at variable names and error codes, which is the most common source of rework. Agree these contracts before the Python code is written.

What does the platform team need to provide for Python workers?

The platform team needs to provide a place to run the worker processes, network access from the workers to the Flowable REST API, and a secure way to store credentials. They also need to set up monitoring for worker health, logging for job outcomes, and a deployment pipeline for worker code that matches how the rest of the platform ships services. Confirm the Flowable version and its REST endpoints in your environment, since the client and the server must be compatible. Also confirm which authentication method the Flowable REST application accepts in your setup, and who rotates the credentials used by workers.

How are retries and failed external worker jobs handled in operations?

A worker that cannot complete a job reports a failure to Flowable, and the engine's retry behavior and job state determine what happens next. Operations teams need a documented rule for each failure type, a way to see jobs that have stopped progressing, and a runbook for reprocessing them. Locks on acquired jobs are coordinated through the database, so a job held by a crashed worker can be picked up again once its lock expires. Confirm the exact retry and lock settings for your Flowable version in the official documentation, and test a crashed worker in a non-production environment before go-live. Monitoring should show failed jobs, age of pending jobs and worker availability.

When is it worth bringing in a partner for Python external workers?

A partner is worth considering when the topic count is high, when the workers touch regulated data or sensitive systems, when the team lacks Flowable or job-queue experience, or when the workers must run reliably across several environments on a deadline. Partners also help when a design review is needed before build work starts, since errors in variable contracts and retry rules are expensive to fix later. DeployOpen plans, implements, hardens and documents self-hosted platforms inside your own infrastructure, and most engagements are fixed-scope with the scope agreed in writing before work starts. Ongoing support is available as a retainer.

Testing and running Python workers in production

Test the Python worker against a local or staging Flowable engine before any production topic is connected to it. Start the engine with the same database type and Flowable version you run in production, deploy a small BPMN model whose External Worker task uses a dedicated test topic, and confirm the worker acquires the job, executes its handler and reports completion. Then force the failure path: raise an exception in the handler and check that the job is reported as failed and that retries behave as your model expects. Exercise a timeout path too, by pausing the handler longer than the lock lasts. Run these checks against the exact client version you plan to deploy, and confirm in the client documentation how it reports failures, how many retries it attempts and what it does with the job after the final attempt, since the Flowable docs do not settle those details for the Python client. The Flowable external client for Python repository is the place to check the current behavior.

The async executor explains why lock handling matters. According to the Flowable advanced documentation, the executor coordinates nodes through lock owners and lock expiration times stored in the database. If a worker holds a job past its lock expiry, the job becomes eligible for another node to pick up, so a slow or crashed Python process can cause the same work to run twice. Make handlers idempotent wherever they call external systems, choose a lock duration that covers your slowest normal execution with margin, and decide how long a handler may run before it should fail the job and return it to Flowable. Confirm with the client how it renews or extends locks during long work, because that behavior determines whether a long-running task is safe.

Monitor job backlog per topic, failed job counts and the age of the oldest acquirable job, and alert when any of them grows outside your normal range. Scale by running more worker processes against the same topic, keeping in mind that each process competes for jobs through the same database locks, so adding workers increases database load as well as throughput. Track the database connection pool and the REST call rate from workers, and confirm with the client whether it supports concurrent acquisition in one process. Keep worker credentials and deployment in the same review as the Flowable configuration so a rotation or upgrade does not silently stop a topic.

How DeployOpen scopes and delivers Python external workers

DeployOpen plans, builds, documents and supports Flowable external workers in Python inside your own infrastructure. We start from a written scope that names the process models, topics, workers and environments involved, then deliver the work in the same controlled way across every stage.

How we scope

Inventory the BPMN models that use the external worker task and the topics each one publishes
Agree which Python services will acquire jobs, and where they run in your environment
Define the Flowable version, authentication method and REST endpoints the workers will call
Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing first

How we build

Implement Python workers on the official Flowable External Worker Client for Python from PyPI
Model retry, failure and completion handling for each topic before workers go into test
Test job acquisition, locking and completion against a non-production Flowable deployment
Keep worker code, configuration and deployment definitions in your version control system

How we hand over

Deliver runbooks covering worker startup, topic configuration, credential rotation and job failure triage
Document how Python workers sit alongside Java delegates and which topics each one serves
Walk your platform and engineering teams through the code, tests and operating procedures
Confirm the handover checklist with your team before closing the engagement

How we support

Offer ongoing support, upgrades and continuous testing for workers as a retainer
Review worker compatibility when you move to a new Flowable release or Python runtime
Confirm support terms, response expectations and access arrangements in the written agreement