When Flowable event logging for audit trails is worth enabling
Flowable event logging can emit engine events as JSON to a queue or search store, giving a regulated team a stream of what the engine did as it happened. It is an addition to Flowable's history, not a replacement for it. History tables record the state and timeline of process instances, case instances and decisions, but they do not answer every question an auditor or security team will ask, and they were not designed to feed a SIEM or a monitoring pipeline in near real time. Enable event logging when you need a reviewable record of who or what changed a process, case or decision, and when that record must leave the engine database for independent retention or analysis.
Teams usually reach for event logging in three situations. The first is an audit question that history cannot answer cleanly, such as the order of administrative actions across several deployments, or evidence that a definition was changed or suspended outside the normal release path. The second is downstream monitoring, where operations needs to see engine activity in near real time without polling the database. The third is SIEM ingestion, where security operations wants engine events alongside application and infrastructure logs. Before you commit, confirm with the Flowable docs for your exact version which event types the engine emits, which fields each carries, and whether the database event logger or a custom flusher fits your target store. Event names and configuration keys differ between major releases, so treat any sample you find elsewhere as a starting point to verify, not a specification.
How event logging differs from Flowable history tables
History tables and event logging answer different audit questions, and an audit team needs to know which one it is relying on before it designs any evidence. History tables hold the state and timeline of process instances, tasks and activities as the engine has stored them. They are written by the engine as part of its own data model and are queried from the database that backs the engine. Event logging works differently. It captures engine events as they happen and hands them to a separate consumer, such as a queue or search store, where they are kept and queried outside the engine's schema. Confirm the history level your deployment uses, since it determines how much the history tables record, and confirm which event types your engine version emits before you map them to a control.
The practical difference shows up in what each source can lose. History tables depend on the engine's transaction and cleanup behavior: rows follow the retention you configure, and any change to the history configuration affects what remains. An event stream depends on the flusher path: if the queue or store is unavailable, or a custom flusher fails silently, events can be missed unless your code retries or fails the transaction. Ask the vendor or your own test environment how events are delivered during a database rollback and during a flusher error, because a rolled-back change may or may not appear in the stream. Test both paths on a copy of your environment rather than assuming the stream matches the tables.
Each source is queried in a different way, and that shapes who can use it. Auditors who need to reconstruct one instance usually query history through the engine's Java or REST APIs or directly from the database, which requires the right access and a documented schema. Operations and security teams who need cross-instance patterns, such as every change to a particular decision across a month, usually prefer the event store, where they can search by the JSON fields the engine writes. Decide early which questions each team asks, record which source answers each one, and agree how the two sources are reconciled, so an auditor does not discover a gap between them during fieldwork.
Which event logging option fits your audit design
This table compares three ways to capture Flowable engine activity: history tables alone, the database event logger, and a custom flusher that sends JSON to a queue or search store. Read each row as a question to answer for your own audit design, then confirm the configuration keys for your target Flowable version in the Advanced and Configuration docs. The main takeaway is that history tables answer process questions, while event streams add a separate store your team must retain, secure and monitor.
| Question | History tables only | Database event logger or custom flusher |
|---|---|---|
| Where do events end up? | Engine history tables in the same relational database as runtime data | Database event logger writes to an event log table; a custom flusher sends JSON to a queue or search store you select |
| Who owns retention? | Your DBA and application team, through history level and cleanup jobs you configure | Your team owns the event store and its retention, separate from engine history cleanup |
| Who controls access to the data? | Anyone with database access to engine tables, so restrict schema grants | The queue or search store's own access model, which you must configure and audit |
| What happens when the downstream store fails? | Engine writes continue to the same database, so audit gaps are rare but depend on history level | Your flusher must define buffering, retry and dead-letter behavior; confirm how the engine behaves when a flush fails |
| Are events ordered and complete? | Ordering follows database writes and history level settings you choose | Ordering and delivery guarantees depend on your queue and flusher design; test duplicate and missing event handling |
| Effort to operate | Lowest: uses existing database backups, monitoring and schema management | Highest: a flusher to build and test, plus broker or search cluster sizing, monitoring and upgrades |
| Schema and version changes | Tied to Flowable history schema and upgrade scripts | Event payload format is yours to version, and must be checked against each Flowable upgrade |
| Confirm before choosing | History level and cleanup settings in the Configuration docs for your version | Event logger keys and custom flusher interfaces in the Advanced docs for your version |
How to turn on event logging and verify the output
This sequence takes a Flowable engine from a supported version and database to a verified stream of JSON event records, with each record traceable to a process instance. It gives the operations team a repeatable check before audit trails depend on the output.
Confirm engine version and database support
Before changing anything, the platform owner records the Flowable engine version, the Spring Boot or Java baseline it runs on, and the database type and version in the target environment. Check these against the Flowable configuration documentation for the deployed version, which lists the supported database types, and note that MySQL versions below 5.6.4 lack millisecond precision, which affects timestamp checks later. Confirm the version in the Flowable release notes if you have moved to Flowable 8.0.0, since that release changed the Spring and JSON baselines. The step is done when a written record names the version, the database and any precision limit that applies.
Choose the event logger or custom flusher
The architecture owner decides between the built-in database event logger and a custom flusher that sends events to a queue or search store. The database logger writes events to a table that the engine owns, while a custom flusher hands events to your own code, which then forwards them. Confirm in the Flowable advanced documentation how each option is registered, how often events are flushed, and what happens to unflushed events if the engine stops. Record the choice and the reason, because the target store and retention rules depend on it. The step is done when the design note states who owns delivery of each event.
Register the logger in engine configuration
A platform engineer adds the chosen event logger or flusher to the engine configuration in the same way as other engine settings, then deploys the change to a non-production environment first. Confirm the configuration keys and class names against the Flowable documentation for the deployed version, since they can change between major releases. Restart the application and check the startup log for any configuration error. The step is done when the engine starts cleanly and the logger is reported as active in the logs or in configuration inspection.
Deploy a test process with start, task and end
A developer deploys a small test process that starts, creates a user task, completes that task and ends. Use a unique business key so the test instance is easy to find later. Run the process through the Java API or the REST application, complete the task, and let the process reach its end event. The step is done when the test process has finished and the engine reports the instance as completed, which gives you a known set of events to look for in the output.
Inspect the JSON output in the target store
The operations engineer opens the queue or search store and finds the events generated by the test instance. Check that each record is valid JSON, that it carries the event type, and that the start, task and completion events all appear. Compare the count of events against the steps the test process executed. The step is done when every expected event type is present once and the JSON parses without errors in the target store.
Check event ordering and timestamp precision
Sort the test events by their recorded time and confirm that they follow the order in which the process ran, from start to task creation to task completion to end. If several events share the same timestamp, check whether the database stores fractional seconds, because the Flowable configuration documentation notes precision limits for some MySQL versions. Record the precision your database supports and confirm that the auditor-facing timestamps are stored in one time zone. The step is done when ordering matches execution and the precision is documented.
Trace a test event to its process instance
Take one event from the test run and confirm that it carries the process instance ID, and that the ID matches the instance shown by the engine API for the test process. Repeat the check for a task event, which should also link to the same instance. This link is what lets an investigator move from an audit record back to the process history. The step is done when every test event for the instance returns the same process instance ID, and the operations team has written down where that ID appears in each event.
Record the verified configuration and rerun the test
Save the working configuration, the engine version, the database precision note and a sample event as the baseline for this environment. Repeat the test after each engine upgrade, database upgrade or change to the flusher, because a change to either can alter event format or ordering. Confirm each step against the Flowable documentation for the deployed version before relying on it. The step is done when the baseline is stored with the environment's operations documentation and the team knows which changes will trigger a retest.
Why the JSON library version matters for event payloads
The JSON library behind Flowable's event output is a change you must plan for, not a detail to discover after an upgrade. Flowable 8.0.0 uses Jackson 3 as its default JSON library, with a Jackson 2 compatibility layer available, and the same release moves the stack to Spring Framework 7 and Spring Boot 4 with no backward compatibility for Spring Boot 3. If your event logging path, custom flusher or parser was written against Jackson 2 behavior, the default change can alter what your consumers receive. Before you choose a version, confirm in the Flowable 8.0.0 release notes which library serializes event payloads in the version you intend to run, and whether the compatibility layer covers your code path.
Payload shape is where drift usually appears. Libraries can differ in how they write date and time values, whether they emit null fields or omit them, how they order properties, and how they handle polymorphic or nested objects. Any of these can change the JSON that reaches a queue or search index even when the engine behaves identically. Ask your team to capture a set of representative events for each process and case type from the current version, then compare them byte for byte against the candidate version. Pay particular attention to timestamp precision and time zone offsets, because audit consumers often sort and correlate on them. If you use a custom flusher, check whether it calls the engine's Jackson classes directly, since those calls will move with the library.
Treat consumer contracts as the test target. Write the schema your downstream systems rely on, including field names, types, nullability and timestamp format, and run contract tests against both the current and upgraded engine in a non-production environment before the change reaches production. Include replay of a sample of historical events through the consumer, because an audit store that silently accepts a changed field can fail a later evidence review. Record the library and version used for serialization in your change record so that the question of which format a given event was written in can be answered later.
Security risks in an audit event stream
An event stream that feeds an audit trail carries process data out of the engine and into systems with their own exposure. The cards below cover the risks a team should plan for before turning on event logging, and the mitigations to decide on in the design rather than after go-live.
Personal data in event payloads
Event payloads can carry process variables, which often include applicant names, account numbers, health or financial details. Once copied into a queue or search index, that data spreads to more systems and retention rules. Before enabling the logger, inventory which variables each process sets, decide which must be excluded or masked, and confirm with your engineers how the chosen event listener or custom flusher builds the JSON.
Access control on the stream
A queue or search store holding engine events can reveal who did what, when, and to which cases. Teams often secure the engine's REST API and forget the downstream copy. Plan separate roles for producers, consumers and administrators, restrict read access to the audit index, and confirm the authentication options your broker or search platform offers before connecting the flusher.
Integrity of stored events
An audit trail is only useful if an investigator can trust that events were not edited after the fact. Ordinary queues and search indexes allow updates and deletes by privileged users. Decide how events will be made tamper-evident, for example by write-once storage, restricted delete rights, or periodic hashing of batches, and confirm which of these your chosen store supports.
Retention and deletion obligations
Event copies may outlive the history tables they mirror, and deletion in the engine does not remove them from the downstream store. Map each event category to a retention period set by your legal and compliance owners, and build a deletion process that reaches every copy. Ask for the retention rules in writing before the stream goes live.
Silent loss of events
If the flusher or the broker drops messages during an outage, the audit trail has a gap that nothing in the engine reports. A failed write can look like normal operation. Decide how failed sends are retried, how a dead-letter path is watched, and how the team reconciles event counts against history tables on a schedule. Test this with a deliberate broker outage before relying on the trail.
Identity data in custom identity services
When identity management is replaced with a custom IdmIdentityService or an LDAP-based implementation, user names, groups and directory attributes can appear in events and copy into the stream. This exposes directory data to systems that were never meant to hold it. Review which identity fields the event payloads include, limit them to the identifiers auditors need, and confirm how your implementation exposes that data.
How event logging changes by deployment model
When Flowable runs as a JAR inside a Java SE application, your code builds the process engine configuration, so the event logger is set on that configuration object before the engine starts. The setting lives in the same version control history as the application, which makes it easy to review and hard to change by accident in one environment.
- Configuration location: Set the event logging option in the bootstrap code that creates the engine, and avoid conditional paths that could skip it in some environments.
- Deployment: Logger changes ship as a new application build. Roll the new JAR to every node in the same window so all nodes write events with the same setup.
- Custom flusher: If you supply your own flusher, package its class in the application artifact and confirm that its client libraries for the queue or search store are on the classpath.
- Clustered check: Each node runs its own logger instance. Confirm that jobs executed on each node produce events and that one process instance does not generate duplicate events from two nodes.
After each release, start a test process on every node and confirm that its events reach the target queue or search store, including after a restart that interrupts buffered output.
In Tomcat or Jetty, the engine usually starts inside the web application, often from a startup listener that builds the process engine. The logger is therefore configured in the web application's own packaging, and its lifecycle follows the deployment. Redeploys are where event loss and duplicate writes usually appear.
- Configuration location: Set the option where the engine is built, typically in a startup listener or configuration class in the WAR. Keep the logger's classes in
WEB-INF/libso the webapp classloader loads them with the engine. - Deployment: Replace the WAR or redeploy through the container manager. Confirm the previous engine shuts down before the new one writes, so two loggers do not send events to the same queue during overlap.
- Shared libraries: If Flowable JARs sit in the container's common library path, one configuration applies to every web application on that server. Decide this deliberately and document which applications share the logger.
- Clustered check: During a rolling restart, confirm no node comes up with event logging disabled. If your flusher adds the node name to each event, check that the field is present so you can trace events to the node that produced them.
After each redeploy, compare event counts for a test process before and after the change, and check the container logs for shutdown errors from the previous engine.
With the Flowable Spring Boot starters, the engine is created as a Spring bean. Event logging is set through application configuration or through a customizer bean that adjusts the engine configuration before startup. Spring profiles and externalized configuration decide what each environment receives, so the logger setup becomes part of the deployment configuration.
- Configuration location: Put the setting in the application configuration files or in a configuration customizer bean. Keep environment-specific values, such as queue endpoints, in externalized configuration rather than in the image.
- Deployment: Build one container image and promote it through environments with different configuration. Confirm that the logger bean is present in the image's startup logs for each environment.
- Actuator scope: The
/actuator/flowableendpoint reports process statistics for monitoring. It does not replace event logging, so confirm separately that events reach the store. Expose the actuator only to monitoring networks. - Clustered check: Run several replicas with the same externalized configuration. Confirm that each replica starts with the same logger settings and that events from each replica reach the store with the replica identity intact.
Before promoting a change, run a test process on two replicas and confirm the events match in type and order across both.
When Flowable runs as the REST application, your team does not write engine startup code. The logger is set through the configuration that the REST application accepts, and any custom flusher needs a build or extension path that Flowable supports. Confirm both with the Flowable documentation for your version before you plan the rollout.
- Configuration location: Identify which settings the REST application exposes for event logging and where they are supplied in your deployment, such as environment variables or mounted configuration files.
- Deployment: Upgrading the REST application means replacing the deployed artifact. Confirm that the event logging configuration survives the replacement and that the new version still loads your flusher.
- Custom flusher: A flusher outside the standard distribution requires your own build of the application. Ask the vendor or check the documentation for the supported extension method before you commit to that build.
- Clustered check: Multiple REST instances behind a load balancer each emit events, and any instance can start a process. Send test requests through the load balancer and confirm that events from every instance appear in the store.
After each upgrade, call the REST endpoints that start, complete and query processes, then check that the corresponding events arrive in order and without gaps.
Limits to confirm before you rely on events as evidence
Flowable's event output is useful for audit trails, but how far you can lean on it depends on details that vary by version, database and deployment. Each item below is a check to run with the Flowable docs or a controlled test in your own environment before an auditor sees the stream.
Confirm which event types ship
Start by listing the event types the deployed Flowable version emits for process, task, job and entity changes. Compare that list with the events your control narrative needs, such as who approved a step, when a variable changed and when a job was retried. Run a short test process, trigger each action you care about, and capture the JSON. Any action without an event needs a second source of evidence, such as application logs or history tables.
Confirm delivery semantics of the flusher
Find out whether the chosen flusher delivers each event at least once or at most once. At least once can produce duplicates after a retry, and at most once can silently drop events after a failure. Test by stopping the downstream queue or search store mid-run, restarting the engine and counting what arrives. Your evidence design should deduplicate on a stable event key if duplicates are possible.
Confirm ordering across cluster nodes
When several engine nodes share one database, events from different nodes can reach the store out of sequence. Ask whether the flusher or the database logger guarantees any ordering, and whether timestamps alone are enough to reconstruct the sequence. Test with two nodes handling the same process instance under load. Plan your audit queries around timestamps and instance identifiers rather than arrival order unless the test proves otherwise.
Check timestamp precision for your database
Event timestamps are only as precise as the column type your database stores. The Flowable configuration docs note that MySQL versions below 5.6.4 lack millisecond precision, so check the version and column type in your environment. Insert events close together on your chosen database and confirm that ordering survives the round trip into your store. Record the precision you verified in the evidence file.
Confirm version and Spring support status
Check the support status of your Flowable version and the Spring baseline it requires before relying on its event output as evidence. Major releases change the Java, Spring and Jakarta baseline, and 8.0.0 requires Spring Framework 7 and Spring Boot 4 without backward compatibility for Spring Boot 3. Ask the vendor which releases receive fixes, and whether event-related defects are covered. Record the version and its end-of-support date in your control documentation.
Test events through upgrades and schema changes
Engine upgrades and database schema changes can alter event formats, table layouts or flusher behavior. Before a production upgrade, replay a representative set of processes on the old and new versions and compare the event streams field by field. Confirm whether events in flight during a rolling restart are flushed, retried or lost. Keep the pre-upgrade stream archived so you can show continuity of evidence across the change.
Questions teams ask before enabling Flowable event logging
How much effort does enabling Flowable event logging take?
Effort depends mostly on three choices: the Flowable engine version you run, whether you use the built-in database event logger or write a custom flusher, and where the events must land. The database logger is the smaller path because it writes events to tables you configure and read. A custom flusher adds code you own: JSON serialization, batching, retries on failed writes, and tests showing that no event is dropped when the downstream store is slow. The downstream store adds its own setup, index or topic design, access control and retention rules. Engine version matters too. Flowable 8.0.0 made Jackson 3 the default JSON library with a Jackson 2 compatibility layer, so a flusher written against one library needs checking against the other. Ask any quote to split effort into engine setup, flusher code, store integration and testing.
Can an in-house team enable Flowable event logging without outside help?
A team with solid Java and SQL skills can enable the database event logger by following the Flowable configuration docs. A custom flusher needs more: working knowledge of the engine's event listener API, Java concurrency for batching and retries, JSON serialization, and the client library for your queue or search store. Someone also has to operate the result, which means watching consumer lag, handling failed writes and planning schema changes during Flowable upgrades. The skill gap usually sits in testing. The team must prove that a rollback, a crash mid-batch or a store outage does not silently lose or duplicate audit events. If those skills exist in-house, the build is manageable. If they are thin, DeployOpen implements, tests and documents event logging inside your infrastructure under a written, fixed-scope agreement, and the quote follows from that scope.
How long does a Flowable event logging project usually take?
The timeline depends on the same drivers as effort, plus approvals and access. The Flowable engine version and deployment model set how much of the setup is standard. A database-only logger feeding a store you already run is the shortest path. A custom flusher, a new queue, network approvals to reach the store and a load test against expected event volume each add steps that need owners. Environment access is often the longest wait in regulated organizations. The most reliable way to get a date is a written scope with named deliverables, followed by a plan agreed with your team. Ask any provider for the timeline as a breakdown by phase, and check which phases depend on your change-approval calendar, test environment availability or store access, since those often control the date more than build work does. No duration applies until a scope is agreed.
What must our team provide before event logging work begins?
Start with a non-production Flowable environment that matches production in engine version, database and deployment model, so event volumes and failure modes can be tested realistically. Add credentials and network paths to the target queue or search store in that environment, plus a documented route to production. Retention decisions are required before build: how long events are kept, what is deleted and when, and whether personal data in event payloads must be masked or excluded. Name a data owner who can approve those decisions and sign off on which fields are logged. Provide a contact on the store platform team and access to current process models, so the event scope matches how processes run today. Missing any of these items usually adds waiting time to the schedule, so confirm each one in the written scope.
What ongoing people time does Flowable event logging require?
Ongoing work is mostly monitoring and change handling, and most organizations do not need a dedicated role for it. Someone watches the flusher for failed writes, growing backlogs in the queue and disk growth in the store. Retention jobs need an owner who confirms deletions match the policy. Each Flowable upgrade needs a check that event payloads still match what the store expects, because serialization libraries and engine fields can change between releases, including the JSON library change in Flowable 8.0.0. Changes to store schemas, indexes or the set of logged events need review by the data owner and the security team. The time required depends on event volume, the number of consumers reading the stream and how often your processes change. Ask any quote to list the operating tasks with their expected frequency, and assign each owner before go-live. Retainers are available for ongoing support.
Does Flowable event logging alone satisfy an audit requirement?
No. Event logging records what the engine did, but an audit tests whether controls operate as designed and whether evidence supports them. Auditors will ask who approved a process change, how access to the event store is restricted, how long logs are kept, whether personal data is protected and whether logs are complete and tamper-resistant. Event logs can answer parts of these questions, but they also need supporting controls such as access reviews, change records, retention rules and documented ownership. DeployOpen prepares systems, controls and evidence for compliance assessments, but it does not issue audit reports, attestations or certifications. Independent auditors and certification bodies make that determination. Confirm the control scope with your auditor before the build, so the logged fields and retention design match the evidence they will request.
How DeployOpen scopes and delivers Flowable event logging
DeployOpen plans, implements, tests and documents Flowable event logging inside your own infrastructure, starting from a written scope that names the events, the destination and the audit questions the stream must answer. The work is delivered as a fixed-scope engagement, with ongoing support available as a retainer once the logger is live.
How we scope
- Inventory the engine events your auditors and security team need, and decide which ones stay in history tables.
- Agree the destination queue or search store, its retention, and who may read the JSON streams.
- Write the scope, deliverables and timeline in advance, so the logging work has a defined end point.
How we build
- Enable the database event logger or write a custom flusher, depending on the throughput and sink requirements.
- Test that every agreed event type reaches the sink with the expected JSON fields and correlation identifiers.
- Check ordering, duplicate handling and behavior when the queue or search store is unavailable during testing.
How we hand over
- Document configuration, the event schema, the flusher code and the data flow for your engineers and auditors.
- Walk your platform and security teams through the operational runbook, including how to verify the stream is flowing.
- Deliver the test evidence and a list of open items that your own teams need to confirm in their environment.
How we support
- Offer ongoing support and upgrades for the logging setup as a retainer, aligned with your Flowable release plan.
- Review the event stream again when the process model, delegates or Flowable version changes.
- Keep the logging configuration under the same change control as the rest of your Flowable deployment.
